Oct 30 18:35:35.174874 ip-10-0-134-58 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:35:35.819584 ip-10-0-134-58 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:35:35.819584 ip-10-0-134-58 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:35:35.819584 ip-10-0-134-58 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:35:35.819584 ip-10-0-134-58 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:35:35.825264 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825196    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:35:35.825264 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825264    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825268    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825271    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825274    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825277    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825279    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825282    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825285    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825288    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825291    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825294    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825296    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825299    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825301    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825304    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825312    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825315    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825317    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825320    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825322    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:35:35.825332 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825325    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825327    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825330    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825333    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825335    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825338    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825341    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825345    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825349    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825352    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825354    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825357    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825360    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825362    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825365    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825367    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825369    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825372    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825374    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:35:35.826048 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825377    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825379    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825382    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825385    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825387    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825389    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825392    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825394    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825435    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825472    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825477    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825481    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825591    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825611    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825620    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825625    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825630    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825640    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825645    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:35:35.826744 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825649    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825654    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825658    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825663    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825667    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825678    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825682    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825686    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825690    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825694    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825699    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825703    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825708    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825713    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825718    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825722    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825727    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825736    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825741    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825746    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:35:35.827242 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825750    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825754    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825759    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825763    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825768    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825772    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825778    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825782    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825787    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825791    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825800    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825804    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825809    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825813    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825817    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825821    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825825    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825830    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825834    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825838    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:35:35.827733 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825842    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825847    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825856    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825860    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825872    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825879    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.825885    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827214    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827224    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827227    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827231    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827233    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827236    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827238    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827241    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827244    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827246    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827249    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827252    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:35:35.828202 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827254    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827257    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827260    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827262    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827265    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827267    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827270    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827272    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827275    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827279    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827283    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827286    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827289    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827292    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827294    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827297    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827299    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827302    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827304    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:35:35.828715 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827307    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827309    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827312    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827315    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827318    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827320    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827323    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827325    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827328    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827330    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827333    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827335    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827338    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827341    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827343    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827346    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827348    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827351    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827354    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827356    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:35:35.829155 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827358    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827361    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827363    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827367    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827372    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827375    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827378    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827381    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827384    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827387    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827389    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827392    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827394    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827397    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827400    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827403    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827406    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827408    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827411    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827413    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:35:35.829677 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827416    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827418    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827420    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827423    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827425    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827428    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827430    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827433    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827435    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827438    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827440    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827444    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827446    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827448    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827451    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827453    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827456    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827458    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827463    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:35:35.830151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827465    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827468    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827470    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827473    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827475    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827478    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827480    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827484    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827486    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827489    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827491    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827494    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827496    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827499    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827501    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.827504    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828024    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828030    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828033    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828036    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:35:35.830604 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828039    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828041    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828044    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828046    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828049    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828052    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828054    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828057    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828059    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828062    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828064    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828069    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828071    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828075    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828077    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828080    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828082    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828085    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828087    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828102    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:35:35.831067 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828106    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828108    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828111    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828115    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828119    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828122    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828125    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828128    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828132    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828136    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828139    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828141    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828145    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828147    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828150    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828152    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828155    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828158    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828160    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:35:35.831530 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828163    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828165    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828168    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828170    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828174    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828177    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828179    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828182    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828185    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828188    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828191    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828194    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828197    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828201    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828203    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828206    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828209    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828211    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828214    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:35:35.831987 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828217    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828219    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828221    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828224    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828226    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828229    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828232    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828234    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828236    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828239    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828241    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828244    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828246    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828249    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828251    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828254    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828256    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828260    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828262    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828265    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:35:35.832441 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828267    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828271    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828274    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828276    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828278    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828281    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828284    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828286    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828289    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828291    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828293    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828296    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828299    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828302    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828304    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828307    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828309    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828311    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828314    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828316    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:35:35.832961 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828319    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828321    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828324    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.828326    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829268    2572 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829278    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829285    2572 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829290    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829295    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829298    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829302    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829306    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829309    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829312    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829317    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829320    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829323    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829327    2572 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829330    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829333    2572 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829336    2572 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829339    2572 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829343    2572 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829346    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:35:35.833411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829350    2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829353    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829356    2572 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829365    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829368    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829371    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829374    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829378    2572 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829381    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829384    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829387    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829390    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829394    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829399    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829402    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829405    2572 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829408    2572 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829412    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829416    2572 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829419    2572 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829422    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829425    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829428    2572 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829437    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:35:35.834087 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829440    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829443    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829446    2572 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829449    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829452    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829454    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829457    2572 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829460    2572 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829463    2572 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829465    2572 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829469    2572 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829472    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829475    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829478    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829481    2572 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829484    2572 flags.go:64] FLAG: --help="false"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829487    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829490    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829493    2572 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829496    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829499    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829502    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829505    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:35:35.834760 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829508    2572 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829511    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829514    2572 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829517    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829520    2572 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829523    2572 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829525    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829528    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829531    2572 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829534    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829537    2572 flags.go:64] FLAG: --lock-file=""
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829540    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829555    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829559    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829564    2572 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829567    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829570    2572 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829573    2572 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829575    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829579    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829582    2572 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829586    2572 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829590    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829594    2572 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829597    2572 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:35:35.835295 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829600    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829603    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829606    2572 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829609    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829612    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829614    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829617    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829625    2572 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829628    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829631    2572 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829633    2572 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829636    2572 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829639    2572 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829642    2572 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829645    2572 flags.go:64] FLAG: --port="10250"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829648    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829651    2572 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-09a53192a76a17ddb"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829654    2572 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829657    2572 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829660    2572 flags.go:64] FLAG: --register-node="true"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829663    2572 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829667    2572 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829669    2572 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829672    2572 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829675    2572 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829679    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:35:35.835911 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829683    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829686    2572 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829691    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829694    2572 flags.go:64] FLAG: --runonce="false"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829697    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829700    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829703    2572 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829706    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829709    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829712    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829715    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829718    2572 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829720    2572 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829723    2572 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829727    2572 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829729    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829732    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829735    2572 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829738    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829743    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829746    2572 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829749    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829753    2572 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829755    2572 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829764    2572 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:35:35.836529 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829767    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829770    2572 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829773    2572 flags.go:64] FLAG: --v="2"
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829777    2572 flags.go:64] FLAG: --version="false"
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829781    2572 flags.go:64] FLAG: --vmodule=""
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829785    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.829788    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829925    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829929    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829934    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829937    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829939    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829942    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829945    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829947    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829949    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829953    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829957    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829961    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829964    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829967    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:35:35.837125 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829970    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829972    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829975    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829977    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829980    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829983    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829985    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829988    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829990    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829992    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829995    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.829998    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830001    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830003    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830005    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830008    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830011    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830013    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830017    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830019    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:35:35.837640 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830023    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830026    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830028    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830031    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830033    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830036    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830038    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830041    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830043    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830046    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830048    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830051    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830053    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830055    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830058    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830060    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830063    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830066    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830068    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:35:35.838113 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830072    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830076    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830079    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830081    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830084    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830087    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830089    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830092    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830094    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830097    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830099    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830103    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830106    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830109    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830112    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830114    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830117    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830119    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830122    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830124    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:35:35.838674 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830127    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830129    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830132    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830134    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830137    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830139    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830142    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830144    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830146    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830149    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830152    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830154    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830157    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830160    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830162    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830165    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830167    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830170    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830173    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:35:35.839143 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830175    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830178    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830181    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830183    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830187    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830189    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830193    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830195    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830198    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830201    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830203    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830205    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830208    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830210    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.830236    2572 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830275    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:35:35.839593 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830279    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830282    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830285    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830288    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830290    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830293    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830295    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830298    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830300    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830303    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830305    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830308    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830310    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830313    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830316    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830318    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830321    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830324    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830326    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:35:35.839963 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830329    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830333    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830335    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830338    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830341    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830344    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830346    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830349    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830351    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830354    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830356    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830359    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830361    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830364    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830366    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830368    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830371    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830373    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830376    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830378    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:35:35.840406 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830381    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830384    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830386    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830390    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830394    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830397    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830399    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830402    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830404    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830407    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830410    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830412    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830415    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830418    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830422    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830425    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830428    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830431    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830434    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830436    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:35:35.840894 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830439    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830441    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830444    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830446    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830448    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830451    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830453    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830456    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830458    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830461    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830463    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830466    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830468    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830471    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830473    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830476    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830479    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830481    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830483    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:35:35.841353 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830486    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830489    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830491    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830493    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830496    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830499    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830502    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830505    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830507    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830511    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830513    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830516    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830519    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830521    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830524    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830526    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830528    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830531    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830533    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830536    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:35:35.841804 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830539    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830541    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830560    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830563    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830565    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830568    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.830573    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.837033    2572 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.837048    2572 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.837062    2572 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.837066    2572 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837120    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837125    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837128    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837132    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837136    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837141    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837144    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837147    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:35:35.842262 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837150    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837154    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837158    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837162    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837165    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837167    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837170    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837172    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837175    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837177    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837180    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837183    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837186    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837188    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837191    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837193    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837196    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837199    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837202    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:35:35.842718 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837204    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837207    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837209    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837212    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837215    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837217    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837220    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837222    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837225    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837227    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837230    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837233    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837235    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837238    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837240    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837243    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837245    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837248    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837251    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837253    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:35:35.843151 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837255    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837258    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837260    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837263    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837266    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837268    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837271    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837274    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837276    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837279    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837282    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837284    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837287    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837289    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837292    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837294    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837296    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837299    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837302    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837304    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:35:35.843678 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837306    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837309    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837311    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837314    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837316    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837319    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837321    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837324    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837326    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837329    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837332    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837334    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837336    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837339    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837341    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837344    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837346    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837349    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837351    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:35:35.844184 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837354    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837357    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837359    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837362    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837365    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837367    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837370    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837372    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837375    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837377    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837380    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837382    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837384    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837387    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837389    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837392    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837394    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837397    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837399    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:35:35.844646 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837402    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.837428    2572 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837597    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837602    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837605    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837608    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837611    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837615    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837618    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837621    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837624    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837626    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837629    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837633    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837635    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837638    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:35:35.845092 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837641    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837644    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837646    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837649    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837652    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837654    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837657    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837659    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837662    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837664    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837667    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837669    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837672    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837674    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837677    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837679    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837682    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837685    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837688    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837690    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:35:35.845458 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837693    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837695    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837698    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837700    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837702    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837705    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837708    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837711    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837714    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837718    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837721    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837724    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837726    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837729    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837731    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837734    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837736    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837739    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837743    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:35:35.845930 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837746    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837749    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837751    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837754    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837756    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837759    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837761    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837763    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837766    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837768    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837771    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837774    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837776    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837779    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837781    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837783    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837786    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837788    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837790    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837793    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:35:35.846374 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837795    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837798    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837800    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837803    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837805    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837808    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837810    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837812    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837815    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837817    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837820    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837822    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837824    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837827    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837829    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837832    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837834    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837836    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837839    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:35:35.846862 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837841    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837843    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837846    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837848    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837851    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837853    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837856    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837858    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837861    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837863    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837865    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837868    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837870    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837873    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.837895    2572 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837940    2572 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:35:35.847288 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837945    2572 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837948    2572 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837951    2572 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837953    2572 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837956    2572 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837958    2572 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837960    2572 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837964    2572 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837968    2572 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837970    2572 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837973    2572 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837976    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837979    2572 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837981    2572 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837983    2572 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837986    2572 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837988    2572 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837992    2572 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837995    2572 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:35:35.847665 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.837998    2572 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838000    2572 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838004    2572 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838006    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838009    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838011    2572 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838014    2572 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838016    2572 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838019    2572 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838021    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838024    2572 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838026    2572 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838028    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838031    2572 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838033    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838036    2572 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838038    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838040    2572 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838043    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838045    2572 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:35:35.848090 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838049    2572 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838052    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838054    2572 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838057    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838059    2572 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838061    2572 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838064    2572 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838066    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838069    2572 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838072    2572 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838074    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838076    2572 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838079    2572 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838081    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838084    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838087    2572 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838089    2572 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838092    2572 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838095    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:35:35.849950 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838097    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838099    2572 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838102    2572 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838104    2572 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838120    2572 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838125    2572 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838128    2572 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838131    2572 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838134    2572 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838136    2572 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838139    2572 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838141    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838144    2572 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838148    2572 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838150    2572 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838153    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838155    2572 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838158    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838160    2572 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:35:35.850650 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838163    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838165    2572 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838168    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838170    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838172    2572 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838175    2572 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838177    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838179    2572 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838182    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838184    2572 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838194    2572 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838197    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838200    2572 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838203    2572 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838205    2572 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838208    2572 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838210    2572 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838212    2572 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838215    2572 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:35:35.851149 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838217    2572 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838220    2572 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838222    2572 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838224    2572 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838227    2572 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838229    2572 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838232    2572 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838235    2572 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:35.838239    2572 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.838292    2572 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.840450    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.841440    2572 server.go:1014] "Starting client certificate rotation"
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.841543    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.841589    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.844854    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:35:35.851685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.846976    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:35:35.862213 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.862190    2572 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:35:35.865786 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.865772    2572 log.go:25] "Validated CRI v1 image API"
Oct 30 18:35:35.866527 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.866503    2572 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:35:35.869291 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.869271    2572 fs.go:135] Filesystem UUIDs: map[1fe97406-82cc-4c68-9e3b-678097457625:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b090eb49-0213-406a-8cfd-4a907394034e:/dev/nvme0n1p3]
Oct 30 18:35:35.869371 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.869291    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 18:35:35.873747 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.873625    2572 manager.go:217] Machine: {Timestamp:2026-10-30 18:35:35.873150741 +0000 UTC m=+0.351765299 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102759 MemoryCapacity:32812011520 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec286d0868f22185312742c111bbe479 SystemUUID:ec286d08-68f2-2185-3127-42c111bbe479 BootID:7050a5f8-ade0-4f7f-bcae-c3d88b552adf Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f2:64:72:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f2:64:72:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:0e:4b:37:5b:30 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812011520 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:35:35.873747 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.873736    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:35:35.873903 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.873827    2572 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:35:35.874912 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.874894    2572 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:35:35.876512 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.876483    2572 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:35:35.876661 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.876514    2572 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-58.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:35:35.876712 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.876664    2572 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:35:35.876712 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.876672    2572 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:35:35.876712 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.876682    2572 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:35:35.877596 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.877586    2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:35:35.878609 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.878599    2572 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:35:35.882753 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.882742    2572 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:35:35.882894 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.882885    2572 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:35:35.882928 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.882922    2572 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:35:35.882957 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.882933    2572 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:35:35.882957 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.882949    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:35:35.883013 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.882961    2572 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:35:35.883013 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.882969    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:35:35.883306 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.883293    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:35:35.885850 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.885835    2572 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:35:35.891966 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.891952    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:35:35.892889 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.892877    2572 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:35:35.892928 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.892906    2572 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:35:35.893061 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.893051    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:35:35.893090 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.893067    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:35:35.893090 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.893073    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:35:35.893090 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.893078    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:35:35.893090 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.893087    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:35:35.893186 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.893095    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:35:35.893943 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.893934    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:35:35.893972 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.893944    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:35:35.893972 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.893950    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:35:35.893972 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.893957    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:35:35.893972 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.893970    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:35:35.894192 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.893979    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:35:35.894192 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.894005    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:35:35.894192 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.894014    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:35:35.898046 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.898032    2572 server.go:1263] "Started kubelet"
Oct 30 18:35:35.898435 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.898302    2572 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:35:35.898871 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.898827    2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:35:35.898965 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.898903    2572 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:35:35.899067 ip-10-0-134-58 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:35:35.900534 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.900516    2572 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:35:35.901676 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.901655    2572 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:35:35.904688 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:35.904668    2572 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 18:35:35.906374 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.906360    2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-58.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 18:35:35.906507 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:35.906490    2572 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 18:35:35.906651 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:35.906635    2572 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-58.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 18:35:35.907380 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:35.906360    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-58.us-east-2.compute.internal.18e3627613985f7d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-58.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-58.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 18:35:35.898005373 +0000 UTC m=+0.376619931,LastTimestamp:2026-10-30 18:35:35.898005373 +0000 UTC m=+0.376619931,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-58.us-east-2.compute.internal,}"
Oct 30 18:35:35.909557 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.909531    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:35:35.910121 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.910102    2572 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:35:35.910226 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:35.910210    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:35.911058 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.911040    2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:35:35.911135 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.911041    2572 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:35:35.911135 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.911076    2572 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:35:35.911222 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.911156    2572 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:35:35.911222 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.911162    2572 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:35:35.911310 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.911237    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:35:35.911310 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.911250    2572 factory.go:55] Registering systemd factory
Oct 30 18:35:35.911310 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.911259    2572 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:35:35.911474 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.911463    2572 factory.go:153] Registering CRI-O factory
Oct 30 18:35:35.911514 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.911476    2572 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:35:35.911514 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.911497    2572 factory.go:103] Registering Raw factory
Oct 30 18:35:35.911514 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.911508    2572 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:35:35.912236 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:35.912196    2572 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-58.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 18:35:35.912852 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.912836    2572 manager.go:319] Starting recovery of all containers
Oct 30 18:35:35.917375 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.917336    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9m9qt"
Oct 30 18:35:35.918197 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:35.918168    2572 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 18:35:35.922078 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.922063    2572 manager.go:324] Recovery completed
Oct 30 18:35:35.923769 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.923751    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9m9qt"
Oct 30 18:35:35.927942 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.927924    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:35:35.930202 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.930182    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:35:35.930281 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.930219    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:35:35.930281 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.930233    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:35:35.930786 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.930774    2572 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:35:35.930786 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.930784    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:35:35.930914 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.930786    2572 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:35:35.930914 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.930860    2572 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:35:35.931511 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.931494    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:35:35.931596 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.931520    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:35:35.931596 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.931530    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:35:35.932172 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:35.932105    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-58.us-east-2.compute.internal.18e362761583a50d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-58.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-58.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-58.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 18:35:35.930201357 +0000 UTC m=+0.408815922,LastTimestamp:2026-10-30 18:35:35.930201357 +0000 UTC m=+0.408815922,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-58.us-east-2.compute.internal,}"
Oct 30 18:35:35.932922 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.932908    2572 policy_none.go:50] "Start"
Oct 30 18:35:35.932988 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.932934    2572 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:35:35.932988 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.932947    2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:35:35.934932 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.934919    2572 policy_none.go:44] "Start"
Oct 30 18:35:35.970717 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.970695    2572 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:35:35.973618 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.973603    2572 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:35:35.974303 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:35.974288    2572 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:35:35.974303 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.974302    2572 server.go:87] "Starting device plugin registration server"
Oct 30 18:35:35.974492 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.974483    2572 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:35:35.974522 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.974494    2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:35:35.974618 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.974602    2572 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:35:35.974688 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.974677    2572 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:35:35.974688 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:35.974686    2572 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:35:35.975127 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:35.975109    2572 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:35:35.975191 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:35.975152    2572 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:36.061782 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.061752    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:35:36.063890 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.063038    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:35:36.063890 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.063056    2572 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:35:36.063890 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.063081    2572 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:35:36.063890 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.063118    2572 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 18:35:36.067108 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.067087    2572 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:35:36.075211 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.075195    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:35:36.075950 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.075934    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:35:36.075950 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.075960    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:35:36.076099 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.075971    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:35:36.076099 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.075992    2572 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.085892 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.085872    2572 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.085979 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.085896    2572 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-58.us-east-2.compute.internal\": node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:36.109281 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.109264    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:36.164228 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.164196    2572 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"]
Oct 30 18:35:36.164320 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.164257    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:35:36.165154 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.165129    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:35:36.165241 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.165162    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:35:36.165241 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.165181    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:35:36.167370 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.167358    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:35:36.167535 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.167521    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.167593 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.167571    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:35:36.168394 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.168377    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:35:36.168469 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.168405    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:35:36.168469 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.168418    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:35:36.168469 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.168452    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:35:36.168594 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.168472    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:35:36.168594 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.168482    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:35:36.170618 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.170604    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.170686 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.170628    2572 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:35:36.171395 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.171380    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:35:36.171468 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.171405    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:35:36.171468 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.171414    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:35:36.209295 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.209280    2572 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found" node="ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.209603 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.209584    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:36.211834 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.211819    2572 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found" node="ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.213020 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.213005    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3cd2cf92111a4cf6f6c288eae5fe1a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"f3cd2cf92111a4cf6f6c288eae5fe1a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.213066 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.213029    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3cd2cf92111a4cf6f6c288eae5fe1a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"f3cd2cf92111a4cf6f6c288eae5fe1a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.213066 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.213051    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/749244aef4ca28f482f48e6f6d30c5f8-config\") pod \"kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"749244aef4ca28f482f48e6f6d30c5f8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.310624 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.310608    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:36.313840 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.313827    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/749244aef4ca28f482f48e6f6d30c5f8-config\") pod \"kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"749244aef4ca28f482f48e6f6d30c5f8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.313896 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.313847    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3cd2cf92111a4cf6f6c288eae5fe1a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"f3cd2cf92111a4cf6f6c288eae5fe1a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.313896 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.313865    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3cd2cf92111a4cf6f6c288eae5fe1a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"f3cd2cf92111a4cf6f6c288eae5fe1a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.313896 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.313889    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3cd2cf92111a4cf6f6c288eae5fe1a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"f3cd2cf92111a4cf6f6c288eae5fe1a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.314025 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.313937    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/749244aef4ca28f482f48e6f6d30c5f8-config\") pod \"kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"749244aef4ca28f482f48e6f6d30c5f8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.314025 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.313941    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3cd2cf92111a4cf6f6c288eae5fe1a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"f3cd2cf92111a4cf6f6c288eae5fe1a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.411299 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.411278    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:36.511702 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.511682    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:36.511790 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.511714    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.514817 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.514798    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:36.612578 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.612414    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:36.712895 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.712875    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:36.813361 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.813341    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:36.841655 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.841641    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 18:35:36.842010 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.841744    2572 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:35:36.913436 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:36.913390    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:36.930033 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.930003    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:30:37 +0000 UTC" deadline="2028-05-02 22:40:59.153241815 +0000 UTC"
Oct 30 18:35:36.930033 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:36.930024    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13204h5m22.223220378s"
Oct 30 18:35:37.014434 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:37.014406    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:37.086955 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:37.086924    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3cd2cf92111a4cf6f6c288eae5fe1a5.slice/crio-faa1dd80e2e92bf9d05e3aea03b5bd5733995ad7888281b0af8b4079958531c0 WatchSource:0}: Error finding container faa1dd80e2e92bf9d05e3aea03b5bd5733995ad7888281b0af8b4079958531c0: Status 404 returned error can't find the container with id faa1dd80e2e92bf9d05e3aea03b5bd5733995ad7888281b0af8b4079958531c0
Oct 30 18:35:37.087266 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:37.087247    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749244aef4ca28f482f48e6f6d30c5f8.slice/crio-8aed94ba983787ab53fa85bf2c419096c9b05b013a9bbd1746f293ec8216d2d0 WatchSource:0}: Error finding container 8aed94ba983787ab53fa85bf2c419096c9b05b013a9bbd1746f293ec8216d2d0: Status 404 returned error can't find the container with id 8aed94ba983787ab53fa85bf2c419096c9b05b013a9bbd1746f293ec8216d2d0
Oct 30 18:35:37.091680 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:37.091662    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:35:37.114989 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:37.114965    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:37.181343 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:37.181299    2572 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:35:37.215621 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:37.215603    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:37.287208 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:37.287190    2572 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:35:37.316434 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:37.316416    2572 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-58.us-east-2.compute.internal\" not found"
Oct 30 18:35:37.362935 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:37.362917    2572 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:35:37.411151 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:37.411133    2572 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:37.422069 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:37.422049    2572 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:35:37.422993 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:37.422981    2572 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:35:37.435639 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:37.435602    2572 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:35:39.069285 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.069253    2572 apiserver.go:52] "Watching apiserver"
Oct 30 18:35:39.077307 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.077273    2572 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:35:39.077691 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.077671    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-kjxws","openshift-cluster-node-tuning-operator/tuned-ktxbz","kube-system/konnectivity-agent-9pnlw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal","openshift-multus/multus-jrtwp","openshift-network-operator/iptables-alerter-62g82","openshift-ovn-kubernetes/ovnkube-node-6hwlk","kube-system/global-pull-secret-syncer-cmf2j","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6","openshift-image-registry/node-ca-5kvjr","openshift-multus/multus-additional-cni-plugins-szvjp","openshift-network-diagnostics/network-check-target-k6kkr","kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal","openshift-dns/node-resolver-6lkjf"]
Oct 30 18:35:39.080155 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.080134    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:35:39.082442 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.082418    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.084538 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.084514    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:35:39.084632 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.084582    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:35:39.084709 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.084688    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lfvmh\""
Oct 30 18:35:39.084709 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.084520    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:35:39.084970 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.084704    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:35:39.085975 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.085946    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:35:39.085975 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.085956    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:35:39.086277 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.086259    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:35:39.086362 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.086277    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:35:39.086362 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.086326    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:35:39.086430 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.086380    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8njxf\""
Oct 30 18:35:39.087026 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.086686    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:35:39.088041 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.087990    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-xr42l\""
Oct 30 18:35:39.088041 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.088006    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:35:39.088210 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.088044    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:35:39.090019 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.090000    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:35:39.090019 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.090018    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-89nks\""
Oct 30 18:35:39.090226 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.090205    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:35:39.090440 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.090424    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.092562 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.092546    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:35:39.093586 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.093555    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:35:39.093677 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.093669    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-s4hwp\""
Oct 30 18:35:39.094689 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.094670    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 18:35:39.095961 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.095542    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.095961 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.095580    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gjz7h\""
Oct 30 18:35:39.095961 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.095699    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:35:39.095961 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.095941    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:35:39.095961 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.095944    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:35:39.098149 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.097884    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:39.098149 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.097952    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:35:39.099406 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.099261    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:35:39.099406 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.099351    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:35:39.099406 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.099385    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sfbzk\""
Oct 30 18:35:39.100129 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.100111    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.102586 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.102567    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:39.102674 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.102636    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:35:39.105705 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.105214    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.108036 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108020    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:39.108190 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108078    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-host\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:35:39.108190 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.108112    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:35:39.108190 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108141    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-serviceca\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:35:39.108190 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108173    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tc4s\" (UniqueName: \"kubernetes.io/projected/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-kube-api-access-7tc4s\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:35:39.108446 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108265    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac85734e-3f97-473f-b3ab-8aadd898f381-dbus\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:39.108446 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108306    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3762f795-944b-458e-9458-c26eaf133871-agent-certs\") pod \"konnectivity-agent-9pnlw\" (UID: \"3762f795-944b-458e-9458-c26eaf133871\") " pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:35:39.108446 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108337    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-system-cni-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.108446 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108366    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-cnibin\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.108446 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108396    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.108685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108456    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-sys-fs\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.108685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108491    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-cnibin\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.108685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108533    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-host-slash\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:35:39.108685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108602    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7563c219-088f-4c66-81f5-86c581f08f34-hosts-file\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:35:39.108685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108653    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-socket-dir-parent\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.108685 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108683    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-device-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.108953 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108737    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7563c219-088f-4c66-81f5-86c581f08f34-tmp-dir\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:35:39.108953 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108765    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-cni-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.108953 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108792    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-k8s-cni-cncf-io\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.108953 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108909    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.109195 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108972    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-iptables-alerter-script\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:35:39.109195 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.108997    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztnjp\" (UniqueName: \"kubernetes.io/projected/7563c219-088f-4c66-81f5-86c581f08f34-kube-api-access-ztnjp\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:35:39.109195 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109024    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-netns\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.109195 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109043    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-cni-multus\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.109195 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109088    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-kubelet\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.109195 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109157    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-hostroot\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.109488 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109194    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-multus-certs\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.109488 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109246    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-socket-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.109488 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109277    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-registration-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.109488 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109327    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpvfc\" (UniqueName: \"kubernetes.io/projected/4c1460ce-789a-411c-b748-2b3506811aa1-kube-api-access-kpvfc\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.109706 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109489    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36d759c0-ed6e-4170-aa8f-70560074a467-cni-binary-copy\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.110187 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109836    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-conf-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.110187 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109874    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-system-cni-dir\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.110187 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109914    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.110187 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109938    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:39.110187 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109960    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-etc-kubernetes\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.110187 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.109982    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac85734e-3f97-473f-b3ab-8aadd898f381-kubelet-config\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:39.110187 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.110004    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/36d759c0-ed6e-4170-aa8f-70560074a467-multus-daemon-config\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.110187 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.110027    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-etc-selinux\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.110187 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.110050    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-cni-binary-copy\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.111153 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.110735    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.111153 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.110768    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7bxj\" (UniqueName: \"kubernetes.io/projected/628d4dcc-30b9-4817-815e-53d36818b7c2-kube-api-access-k7bxj\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.111153 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.110799    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z97l8\" (UniqueName: \"kubernetes.io/projected/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-kube-api-access-z97l8\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:35:39.111153 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.110825    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-cni-bin\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.111153 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.110961    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fn8p\" (UniqueName: \"kubernetes.io/projected/36d759c0-ed6e-4170-aa8f-70560074a467-kube-api-access-6fn8p\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.111153 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.110988    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3762f795-944b-458e-9458-c26eaf133871-konnectivity-ca\") pod \"konnectivity-agent-9pnlw\" (UID: \"3762f795-944b-458e-9458-c26eaf133871\") " pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:35:39.132646 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.132612    2572 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:35:39.132721 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.132689    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xvfq8"
Oct 30 18:35:39.140856 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.140833    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xvfq8"
Oct 30 18:35:39.185194 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.185157    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:35:39.185311 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.185211    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:35:39.185386 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.185315    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:35:39.185449 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.185406    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:35:39.185513 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.185452    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:35:39.186516 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.186498    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:35:39.186607 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.186576    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:35:39.186672 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.186632    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8j87p\""
Oct 30 18:35:39.186672 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.186665    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:35:39.186888 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.186871    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ldjxj\""
Oct 30 18:35:39.186955 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.186940    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:35:39.196895 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.196843    2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:35:39.211389 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211362    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-hostroot\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.211478 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211408    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-socket-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.211478 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211431    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-registration-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.211478 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211447    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpvfc\" (UniqueName: \"kubernetes.io/projected/4c1460ce-789a-411c-b748-2b3506811aa1-kube-api-access-kpvfc\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.211478 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211468    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-conf-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.211635 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211493    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:39.211635 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211511    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-node-log\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.211635 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211533    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-log-socket\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.211635 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211547    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-kubernetes\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.211635 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211569    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.211635 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211584    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.211635 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211600    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-systemd\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.211635 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211625    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac85734e-3f97-473f-b3ab-8aadd898f381-kubelet-config\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211654    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/36d759c0-ed6e-4170-aa8f-70560074a467-multus-daemon-config\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211673    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-cni-netd\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211687    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqjpj\" (UniqueName: \"kubernetes.io/projected/edd196c7-d883-4462-b422-385570ed9169-kube-api-access-tqjpj\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211710    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-systemd\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211736    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-cni-binary-copy\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211750    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211764    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7bxj\" (UniqueName: \"kubernetes.io/projected/628d4dcc-30b9-4817-815e-53d36818b7c2-kube-api-access-k7bxj\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211784    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z97l8\" (UniqueName: \"kubernetes.io/projected/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-kube-api-access-z97l8\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211803    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-cni-bin\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211829    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9ndr\" (UniqueName: \"kubernetes.io/projected/74a6aabd-b72d-422d-ac40-7d07a9e56a41-kube-api-access-z9ndr\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211848    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-netns\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211875    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-multus-certs\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211889    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3762f795-944b-458e-9458-c26eaf133871-konnectivity-ca\") pod \"konnectivity-agent-9pnlw\" (UID: \"3762f795-944b-458e-9458-c26eaf133871\") " pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211903    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-host\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:35:39.211930 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211921    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-serviceca\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:35:39.212520 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211944    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-cnibin\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.212520 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211963    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.212520 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.211977    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-host-slash\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:35:39.212520 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212003    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-k8s-cni-cncf-io\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.212520 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212021    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-etc-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.212520 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212048    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.212520 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212077    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-ovnkube-script-lib\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.212520 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212118    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-lib-modules\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.212520 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212376    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-cni-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.212924 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212615    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-cni-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.212924 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212627    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-hostroot\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.212924 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212665    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-host\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:35:39.212924 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212736    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-socket-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.212924 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212788    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-registration-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.212924 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212794    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.213227 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.212953    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-conf-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.213227 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213041    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-cnibin\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.213227 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213075    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-host-slash\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:35:39.213227 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213089    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac85734e-3f97-473f-b3ab-8aadd898f381-kubelet-config\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:39.213227 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213129    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-ovn\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.213227 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213149    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.213227 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213153    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-sys\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.213227 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213175    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-host\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.213227 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213205    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-iptables-alerter-script\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:35:39.213227 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213233    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-cni-multus\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213257    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-var-lib-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213284    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36d759c0-ed6e-4170-aa8f-70560074a467-cni-binary-copy\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213307    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edd196c7-d883-4462-b422-385570ed9169-tmp\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213334    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-system-cni-dir\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213355    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213045    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213390    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-serviceca\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213461    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-cni-bin\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213496    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-netns\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213528    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-multus-certs\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213572    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/36d759c0-ed6e-4170-aa8f-70560074a467-multus-daemon-config\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213623    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-system-cni-dir\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213641    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-etc-kubernetes\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213650    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-k8s-cni-cncf-io\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.213684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213674    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-kubelet\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213700    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-run-netns\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213762    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213789    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-modprobe-d\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213815    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-run\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213841    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w86dj\" (UniqueName: \"kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj\") pod \"network-check-target-k6kkr\" (UID: \"a7be31dd-dd7b-448e-938b-ba7a0807bf49\") " pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213869    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-etc-selinux\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213954    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fn8p\" (UniqueName: \"kubernetes.io/projected/36d759c0-ed6e-4170-aa8f-70560074a467-kube-api-access-6fn8p\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.213982    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysctl-conf\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214008    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-socket-dir-parent\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214033    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-systemd-units\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214036    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36d759c0-ed6e-4170-aa8f-70560074a467-cni-binary-copy\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214038    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3762f795-944b-458e-9458-c26eaf133871-konnectivity-ca\") pod \"konnectivity-agent-9pnlw\" (UID: \"3762f795-944b-458e-9458-c26eaf133871\") " pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214057    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-slash\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214080    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-etc-kubernetes\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214112    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tc4s\" (UniqueName: \"kubernetes.io/projected/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-kube-api-access-7tc4s\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214188    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-cni-binary-copy\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.214391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214261    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac85734e-3f97-473f-b3ab-8aadd898f381-dbus\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214288    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3762f795-944b-458e-9458-c26eaf133871-agent-certs\") pod \"konnectivity-agent-9pnlw\" (UID: \"3762f795-944b-458e-9458-c26eaf133871\") " pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214314    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-system-cni-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214339    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgqmm\" (UniqueName: \"kubernetes.io/projected/3e7518c9-9163-41ac-9546-556b1773cec6-kube-api-access-vgqmm\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214599    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-etc-selinux\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214604    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-socket-dir-parent\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.214695    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.214760    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret podName:ac85734e-3f97-473f-b3ab-8aadd898f381 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:39.714741412 +0000 UTC m=+3.008807934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret") pod "global-pull-secret-syncer-cmf2j" (UID: "ac85734e-3f97-473f-b3ab-8aadd898f381") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214797    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-sys-fs\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214824    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-cnibin\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214873    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7563c219-088f-4c66-81f5-86c581f08f34-hosts-file\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214936    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-cni-bin\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214947    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-cnibin\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214954    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysctl-d\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214993    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7563c219-088f-4c66-81f5-86c581f08f34-hosts-file\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.214994    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/edd196c7-d883-4462-b422-385570ed9169-etc-tuned\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215031    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-device-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.215143 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215023    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-sys-fs\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215064    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7563c219-088f-4c66-81f5-86c581f08f34-tmp-dir\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215163    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac85734e-3f97-473f-b3ab-8aadd898f381-dbus\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215216    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-system-cni-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215224    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-iptables-alerter-script\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215263    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-cni-multus\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215296    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-ovnkube-config\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215320    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-device-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215322    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-env-overrides\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215351    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e7518c9-9163-41ac-9546-556b1773cec6-ovn-node-metrics-cert\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215396    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysconfig\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215415    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-var-lib-kubelet\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215479    2572 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215492    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215530    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztnjp\" (UniqueName: \"kubernetes.io/projected/7563c219-088f-4c66-81f5-86c581f08f34-kube-api-access-ztnjp\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215555    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-kubelet\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215556    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7563c219-088f-4c66-81f5-86c581f08f34-tmp-dir\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:35:39.215715 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.215694    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-kubelet\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.216474 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.216123    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.219953 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.219929    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3762f795-944b-458e-9458-c26eaf133871-agent-certs\") pod \"konnectivity-agent-9pnlw\" (UID: \"3762f795-944b-458e-9458-c26eaf133871\") " pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:35:39.221616 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.221593    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z97l8\" (UniqueName: \"kubernetes.io/projected/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-kube-api-access-z97l8\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:35:39.222455 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.222429    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpvfc\" (UniqueName: \"kubernetes.io/projected/4c1460ce-789a-411c-b748-2b3506811aa1-kube-api-access-kpvfc\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.223757 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.223734    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tc4s\" (UniqueName: \"kubernetes.io/projected/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-kube-api-access-7tc4s\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:35:39.224238 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.224217    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7bxj\" (UniqueName: \"kubernetes.io/projected/628d4dcc-30b9-4817-815e-53d36818b7c2-kube-api-access-k7bxj\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.225968 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.225945    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fn8p\" (UniqueName: \"kubernetes.io/projected/36d759c0-ed6e-4170-aa8f-70560074a467-kube-api-access-6fn8p\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.227276 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.227253    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztnjp\" (UniqueName: \"kubernetes.io/projected/7563c219-088f-4c66-81f5-86c581f08f34-kube-api-access-ztnjp\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:35:39.258364 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.258300    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal" event={"ID":"f3cd2cf92111a4cf6f6c288eae5fe1a5","Type":"ContainerStarted","Data":"faa1dd80e2e92bf9d05e3aea03b5bd5733995ad7888281b0af8b4079958531c0"}
Oct 30 18:35:39.259801 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.259772    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal" event={"ID":"749244aef4ca28f482f48e6f6d30c5f8","Type":"ContainerStarted","Data":"8aed94ba983787ab53fa85bf2c419096c9b05b013a9bbd1746f293ec8216d2d0"}
Oct 30 18:35:39.316322 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316295    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-var-lib-kubelet\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.316436 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316326    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:39.316436 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316349    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-node-log\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.316436 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316374    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-log-socket\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.316436 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316399    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-kubernetes\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.316436 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316418    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-var-lib-kubelet\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316445    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316461    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-kubernetes\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316477    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-log-socket\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316481    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-systemd\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316499    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-node-log\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.316503    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316518    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-cni-netd\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316535    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-systemd\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316545    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqjpj\" (UniqueName: \"kubernetes.io/projected/edd196c7-d883-4462-b422-385570ed9169-kube-api-access-tqjpj\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316535    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.316559    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs podName:74a6aabd-b72d-422d-ac40-7d07a9e56a41 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:39.816539408 +0000 UTC m=+3.110605914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs") pod "network-metrics-daemon-kjxws" (UID: "74a6aabd-b72d-422d-ac40-7d07a9e56a41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316566    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-cni-netd\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316593    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-systemd\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316614    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9ndr\" (UniqueName: \"kubernetes.io/projected/74a6aabd-b72d-422d-ac40-7d07a9e56a41-kube-api-access-z9ndr\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316655    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-etc-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316679    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.316698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316681    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-systemd\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316701    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-ovnkube-script-lib\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316720    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-etc-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316723    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-lib-modules\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316763    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-ovn\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316781    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-sys\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316803    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316799    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-host\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316844    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-lib-modules\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316862    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-ovn\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316844    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-var-lib-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316887    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-host\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316889    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-sys\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316894    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edd196c7-d883-4462-b422-385570ed9169-tmp\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316922    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-var-lib-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316957    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-kubelet\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.316983    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-run-netns\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317006    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.317416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317029    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-modprobe-d\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317044    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-run\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317061    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w86dj\" (UniqueName: \"kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj\") pod \"network-check-target-k6kkr\" (UID: \"a7be31dd-dd7b-448e-938b-ba7a0807bf49\") " pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317076    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysctl-conf\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317090    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-systemd-units\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317127    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-slash\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317154    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgqmm\" (UniqueName: \"kubernetes.io/projected/3e7518c9-9163-41ac-9546-556b1773cec6-kube-api-access-vgqmm\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317175    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-cni-bin\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317189    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysctl-d\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317212    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/edd196c7-d883-4462-b422-385570ed9169-etc-tuned\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317218    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-kubelet\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317240    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-ovnkube-config\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317253    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-run-netns\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317259    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-env-overrides\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317292    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e7518c9-9163-41ac-9546-556b1773cec6-ovn-node-metrics-cert\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317320    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysconfig\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317389    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysconfig\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.318245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317408    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-ovnkube-script-lib\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.319011 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317425    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.319011 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317525    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-modprobe-d\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.319011 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317568    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-cni-bin\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.319011 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317595    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-env-overrides\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.319011 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317725    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysctl-d\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.319011 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317189    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-run\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.319011 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317746    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysctl-conf\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.319011 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317799    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-slash\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.319011 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.317868    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-systemd-units\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.319011 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.318375    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-ovnkube-config\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.319754 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.319729    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edd196c7-d883-4462-b422-385570ed9169-tmp\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.320226 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.320206    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e7518c9-9163-41ac-9546-556b1773cec6-ovn-node-metrics-cert\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.321735 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.321713    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/edd196c7-d883-4462-b422-385570ed9169-etc-tuned\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.324626 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.324607    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:35:39.324735 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.324630    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:35:39.324735 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.324651    2572 projected.go:196] Error preparing data for projected volume kube-api-access-w86dj for pod openshift-network-diagnostics/network-check-target-k6kkr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:35:39.324735 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.324703    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj podName:a7be31dd-dd7b-448e-938b-ba7a0807bf49 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:39.82468691 +0000 UTC m=+3.118753428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w86dj" (UniqueName: "kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj") pod "network-check-target-k6kkr" (UID: "a7be31dd-dd7b-448e-938b-ba7a0807bf49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:35:39.325987 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.325965    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9ndr\" (UniqueName: \"kubernetes.io/projected/74a6aabd-b72d-422d-ac40-7d07a9e56a41-kube-api-access-z9ndr\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:39.326735 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.326715    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgqmm\" (UniqueName: \"kubernetes.io/projected/3e7518c9-9163-41ac-9546-556b1773cec6-kube-api-access-vgqmm\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.327000 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.326980    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqjpj\" (UniqueName: \"kubernetes.io/projected/edd196c7-d883-4462-b422-385570ed9169-kube-api-access-tqjpj\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.366280 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.366249    2572 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:35:39.394417 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.394397    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:35:39.406042 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:39.406021    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ee3e1b_6db3_4036_b26c_d8ad5d4cc8a3.slice/crio-fda3295ed6547c21104bce3ea46135cc2b26d2e328d9d58fca1a66beea3b5b97 WatchSource:0}: Error finding container fda3295ed6547c21104bce3ea46135cc2b26d2e328d9d58fca1a66beea3b5b97: Status 404 returned error can't find the container with id fda3295ed6547c21104bce3ea46135cc2b26d2e328d9d58fca1a66beea3b5b97
Oct 30 18:35:39.417736 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.417713    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:35:39.423867 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.423847    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:35:39.424949 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.424929    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:35:39.426784 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.426688    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jrtwp"
Oct 30 18:35:39.427764 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:39.427740    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3762f795_944b_458e_9458_c26eaf133871.slice/crio-c3fe00b3d12767cf8289d90c5eaeae0d214b23f5e82fb05d7a22cbd0da0cf4de WatchSource:0}: Error finding container c3fe00b3d12767cf8289d90c5eaeae0d214b23f5e82fb05d7a22cbd0da0cf4de: Status 404 returned error can't find the container with id c3fe00b3d12767cf8289d90c5eaeae0d214b23f5e82fb05d7a22cbd0da0cf4de
Oct 30 18:35:39.435178 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:39.434314    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628d4dcc_30b9_4817_815e_53d36818b7c2.slice/crio-c45e8b0359365cc553682db8d420ee0da63380c7193b24dcf96d959336d76970 WatchSource:0}: Error finding container c45e8b0359365cc553682db8d420ee0da63380c7193b24dcf96d959336d76970: Status 404 returned error can't find the container with id c45e8b0359365cc553682db8d420ee0da63380c7193b24dcf96d959336d76970
Oct 30 18:35:39.440469 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:39.440447    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d759c0_ed6e_4170_aa8f_70560074a467.slice/crio-c21c20d181f19eb90e9a3db0649f647bd4f23ceeaf4fd9e4a9bdde086bb18043 WatchSource:0}: Error finding container c21c20d181f19eb90e9a3db0649f647bd4f23ceeaf4fd9e4a9bdde086bb18043: Status 404 returned error can't find the container with id c21c20d181f19eb90e9a3db0649f647bd4f23ceeaf4fd9e4a9bdde086bb18043
Oct 30 18:35:39.441961 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.441873    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:35:39.452047 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:39.452022    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800f854c_c40c_45d9_9ffa_b7ca3ac38d51.slice/crio-9044b821157b9dd8b87564d99f0be80e50cabac7d4c89cb19da361f37738104c WatchSource:0}: Error finding container 9044b821157b9dd8b87564d99f0be80e50cabac7d4c89cb19da361f37738104c: Status 404 returned error can't find the container with id 9044b821157b9dd8b87564d99f0be80e50cabac7d4c89cb19da361f37738104c
Oct 30 18:35:39.456565 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.456534    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:35:39.459698 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.459667    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:35:39.468370 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:39.468345    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c1460ce_789a_411c_b748_2b3506811aa1.slice/crio-7e71a6838390fe0ef8b0685088c12fec304c859abebc15e65eb0ef7a0f650886 WatchSource:0}: Error finding container 7e71a6838390fe0ef8b0685088c12fec304c859abebc15e65eb0ef7a0f650886: Status 404 returned error can't find the container with id 7e71a6838390fe0ef8b0685088c12fec304c859abebc15e65eb0ef7a0f650886
Oct 30 18:35:39.471676 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:39.471653    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedd196c7_d883_4462_b422_385570ed9169.slice/crio-8b109ede4da66681b8611160701ca49eb8bc4c4d79a95e8461a298a3f835c600 WatchSource:0}: Error finding container 8b109ede4da66681b8611160701ca49eb8bc4c4d79a95e8461a298a3f835c600: Status 404 returned error can't find the container with id 8b109ede4da66681b8611160701ca49eb8bc4c4d79a95e8461a298a3f835c600
Oct 30 18:35:39.482431 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.482401    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:39.492719 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:35:39.492696    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7518c9_9163_41ac_9546_556b1773cec6.slice/crio-05882ddb5a0f5813f31f6de21cef09f1ccbd8629b246f700e925c91beb5340f0 WatchSource:0}: Error finding container 05882ddb5a0f5813f31f6de21cef09f1ccbd8629b246f700e925c91beb5340f0: Status 404 returned error can't find the container with id 05882ddb5a0f5813f31f6de21cef09f1ccbd8629b246f700e925c91beb5340f0
Oct 30 18:35:39.719497 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.719428    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:39.719640 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.719601    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:35:39.719714 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.719673    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret podName:ac85734e-3f97-473f-b3ab-8aadd898f381 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:40.719652899 +0000 UTC m=+4.013719403 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret") pod "global-pull-secret-syncer-cmf2j" (UID: "ac85734e-3f97-473f-b3ab-8aadd898f381") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:35:39.820788 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.820761    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:39.820955 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.820937    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:35:39.821018 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.821008    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs podName:74a6aabd-b72d-422d-ac40-7d07a9e56a41 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:40.820990543 +0000 UTC m=+4.115057043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs") pod "network-metrics-daemon-kjxws" (UID: "74a6aabd-b72d-422d-ac40-7d07a9e56a41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:35:39.922240 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:39.921462    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w86dj\" (UniqueName: \"kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj\") pod \"network-check-target-k6kkr\" (UID: \"a7be31dd-dd7b-448e-938b-ba7a0807bf49\") " pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:39.922240 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.921661    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:35:39.922240 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.921685    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:35:39.922240 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.921698    2572 projected.go:196] Error preparing data for projected volume kube-api-access-w86dj for pod openshift-network-diagnostics/network-check-target-k6kkr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:35:39.922240 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:39.921754    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj podName:a7be31dd-dd7b-448e-938b-ba7a0807bf49 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:40.921736575 +0000 UTC m=+4.215803088 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w86dj" (UniqueName: "kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj") pod "network-check-target-k6kkr" (UID: "a7be31dd-dd7b-448e-938b-ba7a0807bf49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:35:40.144024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.143706    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:30:39 +0000 UTC" deadline="2028-04-01 04:56:54.537295258 +0000 UTC"
Oct 30 18:35:40.144024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.144026    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12442h21m14.393275909s"
Oct 30 18:35:40.275175 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.275049    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5kvjr" event={"ID":"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3","Type":"ContainerStarted","Data":"fda3295ed6547c21104bce3ea46135cc2b26d2e328d9d58fca1a66beea3b5b97"}
Oct 30 18:35:40.285331 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.285298    2572 generic.go:356] "Generic (PLEG): container finished" podID="f3cd2cf92111a4cf6f6c288eae5fe1a5" containerID="c5693126742fbfe3c07fa4c84b9b521b2c4215309736ab1c2fc8bb7010202e48" exitCode=0
Oct 30 18:35:40.285461 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.285395    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal" event={"ID":"f3cd2cf92111a4cf6f6c288eae5fe1a5","Type":"ContainerDied","Data":"c5693126742fbfe3c07fa4c84b9b521b2c4215309736ab1c2fc8bb7010202e48"}
Oct 30 18:35:40.295812 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.295746    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal" event={"ID":"749244aef4ca28f482f48e6f6d30c5f8","Type":"ContainerStarted","Data":"e495d1b3dd75d2ebc46b76ff26b9f2b8e11f197c78870c255e2dc9f50f4865ef"}
Oct 30 18:35:40.298764 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.298714    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6" event={"ID":"4c1460ce-789a-411c-b748-2b3506811aa1","Type":"ContainerStarted","Data":"7e71a6838390fe0ef8b0685088c12fec304c859abebc15e65eb0ef7a0f650886"}
Oct 30 18:35:40.302684 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.302048    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szvjp" event={"ID":"628d4dcc-30b9-4817-815e-53d36818b7c2","Type":"ContainerStarted","Data":"c45e8b0359365cc553682db8d420ee0da63380c7193b24dcf96d959336d76970"}
Oct 30 18:35:40.303436 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.303405    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk" event={"ID":"3e7518c9-9163-41ac-9546-556b1773cec6","Type":"ContainerStarted","Data":"05882ddb5a0f5813f31f6de21cef09f1ccbd8629b246f700e925c91beb5340f0"}
Oct 30 18:35:40.314307 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.314208    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ktxbz" event={"ID":"edd196c7-d883-4462-b422-385570ed9169","Type":"ContainerStarted","Data":"8b109ede4da66681b8611160701ca49eb8bc4c4d79a95e8461a298a3f835c600"}
Oct 30 18:35:40.322431 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.322365    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal" podStartSLOduration=2.3223507899999998 podStartE2EDuration="2.32235079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:35:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:40.321691472 +0000 UTC m=+3.615757996" watchObservedRunningTime="2026-10-30 18:35:40.32235079 +0000 UTC m=+3.616417313"
Oct 30 18:35:40.323397 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.323373    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-62g82" event={"ID":"800f854c-c40c-45d9-9ffa-b7ca3ac38d51","Type":"ContainerStarted","Data":"9044b821157b9dd8b87564d99f0be80e50cabac7d4c89cb19da361f37738104c"}
Oct 30 18:35:40.339872 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.339803    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jrtwp" event={"ID":"36d759c0-ed6e-4170-aa8f-70560074a467","Type":"ContainerStarted","Data":"c21c20d181f19eb90e9a3db0649f647bd4f23ceeaf4fd9e4a9bdde086bb18043"}
Oct 30 18:35:40.341934 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.341900    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6lkjf" event={"ID":"7563c219-088f-4c66-81f5-86c581f08f34","Type":"ContainerStarted","Data":"e7aab5272fcd7c97b395509f63b18d1012b5eec1aeae7c72965a6bce0c3089ae"}
Oct 30 18:35:40.344164 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.344141    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9pnlw" event={"ID":"3762f795-944b-458e-9458-c26eaf133871","Type":"ContainerStarted","Data":"c3fe00b3d12767cf8289d90c5eaeae0d214b23f5e82fb05d7a22cbd0da0cf4de"}
Oct 30 18:35:40.726965 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.726925    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:40.727164 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:40.727143    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:35:40.727236 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:40.727202    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret podName:ac85734e-3f97-473f-b3ab-8aadd898f381 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:42.727184859 +0000 UTC m=+6.021251363 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret") pod "global-pull-secret-syncer-cmf2j" (UID: "ac85734e-3f97-473f-b3ab-8aadd898f381") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:35:40.827605 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.827514    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:40.827745 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:40.827704    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:35:40.827833 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:40.827765    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs podName:74a6aabd-b72d-422d-ac40-7d07a9e56a41 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:42.827746579 +0000 UTC m=+6.121813084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs") pod "network-metrics-daemon-kjxws" (UID: "74a6aabd-b72d-422d-ac40-7d07a9e56a41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:35:40.929187 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:40.928470    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w86dj\" (UniqueName: \"kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj\") pod \"network-check-target-k6kkr\" (UID: \"a7be31dd-dd7b-448e-938b-ba7a0807bf49\") " pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:40.929187 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:40.928617    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:35:40.929187 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:40.928647    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:35:40.929187 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:40.928661    2572 projected.go:196] Error preparing data for projected volume kube-api-access-w86dj for pod openshift-network-diagnostics/network-check-target-k6kkr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:35:40.929187 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:40.928712    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj podName:a7be31dd-dd7b-448e-938b-ba7a0807bf49 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:42.928695953 +0000 UTC m=+6.222762467 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w86dj" (UniqueName: "kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj") pod "network-check-target-k6kkr" (UID: "a7be31dd-dd7b-448e-938b-ba7a0807bf49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:35:41.145108 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:41.145032    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:30:39 +0000 UTC" deadline="2028-04-09 03:32:39.312150148 +0000 UTC"
Oct 30 18:35:41.145108 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:41.145107    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12632h56m58.167062851s"
Oct 30 18:35:41.254276 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:41.254250    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:41.254405 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:41.254347    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:35:41.254703 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:41.254638    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:41.254763 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:41.254711    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:35:41.254763 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:41.254755    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:41.254840 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:41.254801    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:35:41.352323 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:41.352152    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal" event={"ID":"f3cd2cf92111a4cf6f6c288eae5fe1a5","Type":"ContainerStarted","Data":"eab93ead3eab96d49f3683028c7ed7c1151005863fa6a277a03cb1b364df2000"}
Oct 30 18:35:42.364174 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:42.364049    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szvjp" event={"ID":"628d4dcc-30b9-4817-815e-53d36818b7c2","Type":"ContainerStarted","Data":"bdf711f80f0dfaf196b2d41346fe428f6be0947244a5357a377f0059fffd35e6"}
Oct 30 18:35:42.385354 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:42.384361    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal" podStartSLOduration=4.384344006 podStartE2EDuration="4.384344006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:35:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:41.367452195 +0000 UTC m=+4.661518740" watchObservedRunningTime="2026-10-30 18:35:42.384344006 +0000 UTC m=+5.678410531"
Oct 30 18:35:42.742548 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:42.742509    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:42.742755 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:42.742681    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:35:42.742755 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:42.742743    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret podName:ac85734e-3f97-473f-b3ab-8aadd898f381 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:46.742726826 +0000 UTC m=+10.036793325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret") pod "global-pull-secret-syncer-cmf2j" (UID: "ac85734e-3f97-473f-b3ab-8aadd898f381") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:35:42.843169 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:42.843133    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:42.843327 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:42.843274    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:35:42.843402 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:42.843332    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs podName:74a6aabd-b72d-422d-ac40-7d07a9e56a41 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:46.843315796 +0000 UTC m=+10.137382296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs") pod "network-metrics-daemon-kjxws" (UID: "74a6aabd-b72d-422d-ac40-7d07a9e56a41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:35:42.943646 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:42.943597    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w86dj\" (UniqueName: \"kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj\") pod \"network-check-target-k6kkr\" (UID: \"a7be31dd-dd7b-448e-938b-ba7a0807bf49\") " pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:42.943818 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:42.943768    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:35:42.943818 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:42.943786    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:35:42.943818 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:42.943797    2572 projected.go:196] Error preparing data for projected volume kube-api-access-w86dj for pod openshift-network-diagnostics/network-check-target-k6kkr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:35:42.943972 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:42.943852    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj podName:a7be31dd-dd7b-448e-938b-ba7a0807bf49 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:46.943835859 +0000 UTC m=+10.237902376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w86dj" (UniqueName: "kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj") pod "network-check-target-k6kkr" (UID: "a7be31dd-dd7b-448e-938b-ba7a0807bf49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:35:43.250637 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:43.250605    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:43.250758 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:43.250730    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:35:43.251111 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:43.251081    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:43.251379 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:43.251189    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:35:43.251379 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:43.251237    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:43.251379 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:43.251326    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:35:43.372969 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:43.372935    2572 generic.go:356] "Generic (PLEG): container finished" podID="628d4dcc-30b9-4817-815e-53d36818b7c2" containerID="bdf711f80f0dfaf196b2d41346fe428f6be0947244a5357a377f0059fffd35e6" exitCode=0
Oct 30 18:35:43.373479 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:43.372986    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szvjp" event={"ID":"628d4dcc-30b9-4817-815e-53d36818b7c2","Type":"ContainerDied","Data":"bdf711f80f0dfaf196b2d41346fe428f6be0947244a5357a377f0059fffd35e6"}
Oct 30 18:35:44.381411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:44.381325    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9pnlw" event={"ID":"3762f795-944b-458e-9458-c26eaf133871","Type":"ContainerStarted","Data":"38366d4d4cdd4b9d29fd092042b021597ba50c62388bd9b1c9adb52571fb79d8"}
Oct 30 18:35:44.383900 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:44.383874    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5kvjr" event={"ID":"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3","Type":"ContainerStarted","Data":"f02d08512635f64fc1e8134f85e6aebdde45deb74fefebd4a1804a29185abf38"}
Oct 30 18:35:44.416294 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:44.416217    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9pnlw" podStartSLOduration=3.659218493 podStartE2EDuration="7.416198553s" totalImagesPullingTime="3.75698006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:44.40008132 +0000 UTC m=+7.694147842" watchObservedRunningTime="2026-10-30 18:35:44.416198553 +0000 UTC m=+7.710265075"
Oct 30 18:35:44.768391 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:44.768360    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:35:45.248744 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:45.248708    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:45.248915 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:45.248843    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:35:45.249412 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:45.249234    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:45.249412 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:45.249260    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:45.249412 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:45.249371    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:35:45.249649 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:45.249465    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:35:45.393828 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:45.393752    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6" event={"ID":"4c1460ce-789a-411c-b748-2b3506811aa1","Type":"ContainerStarted","Data":"7d3428209852edd51aab5f33e395c4326c858646b3281d00f051af7aae4059bf"}
Oct 30 18:35:45.796766 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:45.796618    2572 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:35:45.797269 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:45.797073    2572 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:35:45.821246 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:45.820605    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5kvjr" podStartSLOduration=4.314502738 podStartE2EDuration="8.82057856s" totalImagesPullingTime="4.506075822s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:44.416597969 +0000 UTC m=+7.710664469" watchObservedRunningTime="2026-10-30 18:35:45.82057856 +0000 UTC m=+9.114645084"
Oct 30 18:35:46.397982 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:46.397945    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:35:46.601963 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:46.601937    2572 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:35:46.779208 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:46.779171    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:46.779399 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:46.779336    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:35:46.779467 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:46.779410    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret podName:ac85734e-3f97-473f-b3ab-8aadd898f381 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:54.779388734 +0000 UTC m=+18.073455250 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret") pod "global-pull-secret-syncer-cmf2j" (UID: "ac85734e-3f97-473f-b3ab-8aadd898f381") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:35:46.880110 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:46.880058    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:46.880306 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:46.880285    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:35:46.880371 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:46.880359    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs podName:74a6aabd-b72d-422d-ac40-7d07a9e56a41 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:54.880341662 +0000 UTC m=+18.174408184 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs") pod "network-metrics-daemon-kjxws" (UID: "74a6aabd-b72d-422d-ac40-7d07a9e56a41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:35:46.981354 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:46.981320    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w86dj\" (UniqueName: \"kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj\") pod \"network-check-target-k6kkr\" (UID: \"a7be31dd-dd7b-448e-938b-ba7a0807bf49\") " pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:46.981545 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:46.981525    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:35:46.981624 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:46.981554    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:35:46.981624 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:46.981568    2572 projected.go:196] Error preparing data for projected volume kube-api-access-w86dj for pod openshift-network-diagnostics/network-check-target-k6kkr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:35:46.981726 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:46.981626    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj podName:a7be31dd-dd7b-448e-938b-ba7a0807bf49 nodeName:}" failed. No retries permitted until 2026-10-30 18:35:54.98161353 +0000 UTC m=+18.275680030 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w86dj" (UniqueName: "kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj") pod "network-check-target-k6kkr" (UID: "a7be31dd-dd7b-448e-938b-ba7a0807bf49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:35:47.166035 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:47.165940    2572 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:35:46.601961148Z","UUID":"ad499f0f-45c6-4d24-9e10-7d212c5c81d9","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:35:47.168334 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:47.168313    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:35:47.168334 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:47.168344    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:35:47.250297 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:47.249852    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:47.250297 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:47.249897    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:47.250297 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:47.249963    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:35:47.250297 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:47.250076    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:35:47.250297 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:47.250131    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:47.250297 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:47.250218    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:35:47.408928 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:47.408890    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6" event={"ID":"4c1460ce-789a-411c-b748-2b3506811aa1","Type":"ContainerStarted","Data":"6ff0e78a606ff07e3bae69e877a37b11dd11d25a04bd15b695e9f92b364f376a"}
Oct 30 18:35:47.411906 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:47.411877    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jrtwp" event={"ID":"36d759c0-ed6e-4170-aa8f-70560074a467","Type":"ContainerStarted","Data":"f1a2588d0817a1ed110f280adf8ba1fe17bb43610b578e6c436d34a72039f4e2"}
Oct 30 18:35:47.428452 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:47.428405    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-jrtwp" podStartSLOduration=2.81328375 podStartE2EDuration="10.428390165s" totalImagesPullingTime="7.615106415s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:47.427557402 +0000 UTC m=+10.721623942" watchObservedRunningTime="2026-10-30 18:35:47.428390165 +0000 UTC m=+10.722456681"
Oct 30 18:35:48.415655 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:48.415431    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6lkjf" event={"ID":"7563c219-088f-4c66-81f5-86c581f08f34","Type":"ContainerStarted","Data":"675a8b293d4906e8f1edfa91ece2d448958ccc6a4093349ce06484848ee593da"}
Oct 30 18:35:48.418271 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:48.418244    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6" event={"ID":"4c1460ce-789a-411c-b748-2b3506811aa1","Type":"ContainerStarted","Data":"12913898436ee6652bd4ce71066bdb4b1bbabdff3abf61a64a13118798062767"}
Oct 30 18:35:48.421314 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:48.421284    2572 generic.go:356] "Generic (PLEG): container finished" podID="628d4dcc-30b9-4817-815e-53d36818b7c2" containerID="e6dc8799f37c897422563741be9ca838a52d5411d873a163ed2f7791671ff8c1" exitCode=0
Oct 30 18:35:48.421428 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:48.421362    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szvjp" event={"ID":"628d4dcc-30b9-4817-815e-53d36818b7c2","Type":"ContainerDied","Data":"e6dc8799f37c897422563741be9ca838a52d5411d873a163ed2f7791671ff8c1"}
Oct 30 18:35:48.429021 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:48.428974    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-6lkjf" podStartSLOduration=3.545660887 podStartE2EDuration="11.428960512s" totalImagesPullingTime="7.883299625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:48.428400266 +0000 UTC m=+11.722466812" watchObservedRunningTime="2026-10-30 18:35:48.428960512 +0000 UTC m=+11.723027037"
Oct 30 18:35:48.444243 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:48.444210    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6" podStartSLOduration=3.517981813 podStartE2EDuration="11.44419922s" totalImagesPullingTime="7.926217407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:35:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:48.443813023 +0000 UTC m=+11.737879547" watchObservedRunningTime="2026-10-30 18:35:48.44419922 +0000 UTC m=+11.738265745"
Oct 30 18:35:49.248745 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:49.248193    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:49.248745 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:49.248260    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:49.248745 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:49.248262    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:49.248745 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:49.248379    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:35:49.248745 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:49.248566    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:35:49.248745 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:49.248653    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:35:49.424579 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:49.424550    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-62g82" event={"ID":"800f854c-c40c-45d9-9ffa-b7ca3ac38d51","Type":"ContainerStarted","Data":"01d8e174355096c3a35eec1c2b87c6aeeaf88a2914e46169b5b1525f37e52e9b"}
Oct 30 18:35:49.444526 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:49.444471    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-62g82" podStartSLOduration=4.823464882 podStartE2EDuration="12.444453345s" totalImagesPullingTime="7.620988463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:49.440861905 +0000 UTC m=+12.734928440" watchObservedRunningTime="2026-10-30 18:35:49.444453345 +0000 UTC m=+12.738519867"
Oct 30 18:35:50.428128 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:50.428073    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ktxbz" event={"ID":"edd196c7-d883-4462-b422-385570ed9169","Type":"ContainerStarted","Data":"49b8ca0df2dd16de38a0529f06d51395ac120fefc2127a6cf3b08c9802717ca3"}
Oct 30 18:35:50.430887 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:50.430856    2572 generic.go:356] "Generic (PLEG): container finished" podID="628d4dcc-30b9-4817-815e-53d36818b7c2" containerID="5bc83cb525d355860c03050a55f5a5ef5cee7c6198b2c588f21b84489b107f57" exitCode=0
Oct 30 18:35:50.431007 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:50.430936    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szvjp" event={"ID":"628d4dcc-30b9-4817-815e-53d36818b7c2","Type":"ContainerDied","Data":"5bc83cb525d355860c03050a55f5a5ef5cee7c6198b2c588f21b84489b107f57"}
Oct 30 18:35:50.444979 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:50.444933    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ktxbz" podStartSLOduration=3.502440591 podStartE2EDuration="13.444916586s" totalImagesPullingTime="9.942475995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:50.444797383 +0000 UTC m=+13.738863918" watchObservedRunningTime="2026-10-30 18:35:50.444916586 +0000 UTC m=+13.738983109"
Oct 30 18:35:51.248175 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:51.248146    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:51.248175 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:51.248165    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:51.248411 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:51.248283    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:35:51.248411 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:51.248344    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:35:51.248411 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:51.248366    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:51.248544 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:51.248434    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:35:52.441119 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:52.440911    2572 generic.go:356] "Generic (PLEG): container finished" podID="628d4dcc-30b9-4817-815e-53d36818b7c2" containerID="24ccaaac8c0e953005ccba478e23eef967fbfd3df6b26c8f9cecaba817043fd0" exitCode=0
Oct 30 18:35:52.441119 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:52.440956    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szvjp" event={"ID":"628d4dcc-30b9-4817-815e-53d36818b7c2","Type":"ContainerDied","Data":"24ccaaac8c0e953005ccba478e23eef967fbfd3df6b26c8f9cecaba817043fd0"}
Oct 30 18:35:53.248429 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:53.248390    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:53.248615 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:53.248452    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:53.248615 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:53.248457    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:53.248615 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:53.248548    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:35:53.248762 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:53.248674    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:35:53.248762 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:53.248747    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:35:54.450865 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:54.450676    2572 generic.go:356] "Generic (PLEG): container finished" podID="628d4dcc-30b9-4817-815e-53d36818b7c2" containerID="f05cc31a335413ada85cb1b294e5aa9025f681cab032e0ed28c885249900378e" exitCode=0
Oct 30 18:35:54.451278 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:54.450751    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szvjp" event={"ID":"628d4dcc-30b9-4817-815e-53d36818b7c2","Type":"ContainerDied","Data":"f05cc31a335413ada85cb1b294e5aa9025f681cab032e0ed28c885249900378e"}
Oct 30 18:35:54.454011 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:54.453990    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk" event={"ID":"3e7518c9-9163-41ac-9546-556b1773cec6","Type":"ContainerStarted","Data":"ff787eab8761bf808a45af77d3be044870838e1dd3fe0e6e17b4a9d6ab8485db"}
Oct 30 18:35:54.454122 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:54.454061    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk" event={"ID":"3e7518c9-9163-41ac-9546-556b1773cec6","Type":"ContainerStarted","Data":"1559253533b1303094195aa3ec49019d0450feae89abf29907b7cbf6ada63dba"}
Oct 30 18:35:54.454192 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:54.454129    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk" event={"ID":"3e7518c9-9163-41ac-9546-556b1773cec6","Type":"ContainerStarted","Data":"d1275e28d8f97bd5bef108d5b5a6244a208a415ce5ab3fa2a09b71633634c95c"}
Oct 30 18:35:54.454192 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:54.454179    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk" event={"ID":"3e7518c9-9163-41ac-9546-556b1773cec6","Type":"ContainerStarted","Data":"34226dd40b85787a80328bbce36a78601deaf0cd8c9fac38f7e924f444330192"}
Oct 30 18:35:54.454292 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:54.454228    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk" event={"ID":"3e7518c9-9163-41ac-9546-556b1773cec6","Type":"ContainerStarted","Data":"9cbded947d6fb74b2e390f1585b0408c1960a6f9c2e441c97c5737400b02e8b3"}
Oct 30 18:35:54.454292 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:54.454276    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk" event={"ID":"3e7518c9-9163-41ac-9546-556b1773cec6","Type":"ContainerStarted","Data":"90257bd7e082ae2739e87774a093e241eeccd0f1995452cbf5e79fc87b3a3a15"}
Oct 30 18:35:54.839505 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:54.839422    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:54.839639 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:54.839568    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:35:54.839709 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:54.839641    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret podName:ac85734e-3f97-473f-b3ab-8aadd898f381 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.839621846 +0000 UTC m=+34.133688347 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret") pod "global-pull-secret-syncer-cmf2j" (UID: "ac85734e-3f97-473f-b3ab-8aadd898f381") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:35:54.940290 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:54.940257    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:54.940420 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:54.940362    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:35:54.940420 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:54.940413    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs podName:74a6aabd-b72d-422d-ac40-7d07a9e56a41 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.940399189 +0000 UTC m=+34.234465689 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs") pod "network-metrics-daemon-kjxws" (UID: "74a6aabd-b72d-422d-ac40-7d07a9e56a41") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:35:55.040664 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:55.040635    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w86dj\" (UniqueName: \"kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj\") pod \"network-check-target-k6kkr\" (UID: \"a7be31dd-dd7b-448e-938b-ba7a0807bf49\") " pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:55.040789 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:55.040773    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:35:55.040839 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:55.040792    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:35:55.040839 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:55.040801    2572 projected.go:196] Error preparing data for projected volume kube-api-access-w86dj for pod openshift-network-diagnostics/network-check-target-k6kkr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:35:55.040903 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:55.040841    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj podName:a7be31dd-dd7b-448e-938b-ba7a0807bf49 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:11.040830172 +0000 UTC m=+34.334896699 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w86dj" (UniqueName: "kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj") pod "network-check-target-k6kkr" (UID: "a7be31dd-dd7b-448e-938b-ba7a0807bf49") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:35:55.248438 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:55.248414    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:55.248438 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:55.248433    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:55.248633 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:55.248463    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:55.248633 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:55.248539    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:35:55.248732 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:55.248633    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:35:55.248763 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:55.248734    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:35:55.459336 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:55.459310    2572 generic.go:356] "Generic (PLEG): container finished" podID="628d4dcc-30b9-4817-815e-53d36818b7c2" containerID="39b150b284673cfc3fc7e791833a88b09189820c03a7ed795f8d4b0c6b60f8ff" exitCode=0
Oct 30 18:35:55.459665 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:55.459364    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szvjp" event={"ID":"628d4dcc-30b9-4817-815e-53d36818b7c2","Type":"ContainerDied","Data":"39b150b284673cfc3fc7e791833a88b09189820c03a7ed795f8d4b0c6b60f8ff"}
Oct 30 18:35:56.464879 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:56.464846    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-szvjp" event={"ID":"628d4dcc-30b9-4817-815e-53d36818b7c2","Type":"ContainerStarted","Data":"81c5191c5748206ee15d1856a7b33f64ca7a6b9642a2b689eed0d82aa06a8b8e"}
Oct 30 18:35:56.467933 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:56.467911    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk" event={"ID":"3e7518c9-9163-41ac-9546-556b1773cec6","Type":"ContainerStarted","Data":"980e8573a3d2b103b928d01768b6ef5c88ca3552107912765cfe73e71b93d33f"}
Oct 30 18:35:56.485535 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:56.485495    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-szvjp" podStartSLOduration=8.761014531 podStartE2EDuration="19.485482776s" totalImagesPullingTime="10.724468245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:35:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:56.483704124 +0000 UTC m=+19.777770646" watchObservedRunningTime="2026-10-30 18:35:56.485482776 +0000 UTC m=+19.779549299"
Oct 30 18:35:57.248535 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:57.248498    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:57.248674 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:57.248598    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:35:57.248736 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:57.248678    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:57.248793 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:57.248770    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:35:57.248870 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:57.248831    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:57.248918 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:57.248902    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:35:59.248492 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:59.248459    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:35:59.249246 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:59.248558    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:35:59.249246 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:59.248636    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:35:59.249246 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:59.248636    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:35:59.249246 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:59.248726    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:35:59.249246 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:35:59.248796    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:35:59.477874 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:59.477845    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk" event={"ID":"3e7518c9-9163-41ac-9546-556b1773cec6","Type":"ContainerStarted","Data":"abccd1335fb6784fc8969466c29ed939ddfbcdc7382c7ab12c71b429733ec60a"}
Oct 30 18:35:59.478121 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:59.478089    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:59.478214 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:59.478189    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:59.494123 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:59.494085    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:35:59.501687 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:35:59.501617    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk" podStartSLOduration=8.058788105 podStartE2EDuration="22.501607463s" totalImagesPullingTime="14.442819358s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:35:59.499766883 +0000 UTC m=+22.793833405" watchObservedRunningTime="2026-10-30 18:35:59.501607463 +0000 UTC m=+22.795673985"
Oct 30 18:36:00.358864 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:00.358648    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cmf2j"]
Oct 30 18:36:00.359304 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:00.358983    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:00.359304 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:00.359071    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:36:00.362454 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:00.362430    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kjxws"]
Oct 30 18:36:00.362598 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:00.362487    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k6kkr"]
Oct 30 18:36:00.362598 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:00.362551    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:00.362691 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:00.362618    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:36:00.362691 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:00.362661    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:00.362770 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:00.362710    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:36:00.480611 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:00.480581    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:00.495258 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:00.495239    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:02.248803 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:02.248773    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:02.248803 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:02.248802    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:02.249327 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:02.248862    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:36:02.249327 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:02.248883    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:02.249327 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:02.248955    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:36:02.249327 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:02.249031    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:36:04.248395 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:04.248362    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:04.248395 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:04.248362    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:04.248932 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:04.248536    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:36:04.248932 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:04.248578    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:04.248932 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:04.248663    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:36:04.248932 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:04.248729    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:36:06.248146 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.248119    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:06.248146 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.248127    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:06.248600 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.248127    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:06.248600 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:06.248211    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k6kkr" podUID="a7be31dd-dd7b-448e-938b-ba7a0807bf49"
Oct 30 18:36:06.248600 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:06.248295    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cmf2j" podUID="ac85734e-3f97-473f-b3ab-8aadd898f381"
Oct 30 18:36:06.248600 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:06.248387    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kjxws" podUID="74a6aabd-b72d-422d-ac40-7d07a9e56a41"
Oct 30 18:36:06.793296 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.793274    2572 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeReady"
Oct 30 18:36:06.793440 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.793374    2572 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 18:36:06.836480 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.836455    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-985h5"]
Oct 30 18:36:06.840282 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.840264    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-985h5"
Oct 30 18:36:06.842471 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.842448    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6684c84797-p6964"]
Oct 30 18:36:06.845780 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.845754    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 18:36:06.846514 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.846494    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-tfwdw"]
Oct 30 18:36:06.851264 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.851243    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:06.851400 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.851378    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-tfwdw"
Oct 30 18:36:06.851511 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.851244    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz"]
Oct 30 18:36:06.855771 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.855748    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-v788b"]
Oct 30 18:36:06.855852 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.855813    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz"
Oct 30 18:36:06.859993 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.859973    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"]
Oct 30 18:36:06.860084 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.860070    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-v788b"
Oct 30 18:36:06.863398 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.863379    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 18:36:06.864331 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.864309    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"]
Oct 30 18:36:06.865175 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.865161    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:06.868035 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.868014    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:06.868152 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.867831    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-qhk9m"]
Oct 30 18:36:06.873414 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.873390    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"]
Oct 30 18:36:06.876519 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.876500    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-68flm"]
Oct 30 18:36:06.877657 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.877639    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:06.878247 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.877838    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:06.880675 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.880658    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"]
Oct 30 18:36:06.882652 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.880834    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:06.884447 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.884406    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"]
Oct 30 18:36:06.884551 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.884529    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:06.894674 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.894647    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76f45597c6-hfv8h"]
Oct 30 18:36:06.895528 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.895511    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:06.899881 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.899858    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-59c4cc76f-fbdqx"]
Oct 30 18:36:06.899991 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.899972    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:06.903073 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.903049    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n"]
Oct 30 18:36:06.903362 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.903342    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:06.906795 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.906779    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-985h5"]
Oct 30 18:36:06.906904 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.906810    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-tfwdw"]
Oct 30 18:36:06.906904 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.906834    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jcn7p"]
Oct 30 18:36:06.906904 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.906875    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n"
Oct 30 18:36:06.910272 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.910253    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"]
Oct 30 18:36:06.910374 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.910288    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"]
Oct 30 18:36:06.910374 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.910318    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z2nsf"]
Oct 30 18:36:06.910374 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.910339    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:06.913351 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913336    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"]
Oct 30 18:36:06.913424 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913360    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"]
Oct 30 18:36:06.913424 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913372    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-68flm"]
Oct 30 18:36:06.913424 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913386    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-v788b"]
Oct 30 18:36:06.913424 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913403    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76f45597c6-hfv8h"]
Oct 30 18:36:06.913424 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913416    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6684c84797-p6964"]
Oct 30 18:36:06.913564 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913428    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-qhk9m"]
Oct 30 18:36:06.913564 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913441    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jcn7p"]
Oct 30 18:36:06.913564 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913449    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z2nsf"
Oct 30 18:36:06.913564 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913458    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n"]
Oct 30 18:36:06.913564 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913471    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59c4cc76f-fbdqx"]
Oct 30 18:36:06.913564 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913485    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z2nsf"]
Oct 30 18:36:06.913564 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913502    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"]
Oct 30 18:36:06.913564 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.913518    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz"]
Oct 30 18:36:06.930162 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.930145    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e32eb376-a16f-421c-8958-a410a5650964-ca-trust-extracted\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:06.930258 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.930172    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd89g\" (UniqueName: \"kubernetes.io/projected/b0939980-01bb-4ae3-9c95-5e190b719cbe-kube-api-access-jd89g\") pod \"kas-connection-checker-685895c7c9-985h5\" (UID: \"b0939980-01bb-4ae3-9c95-5e190b719cbe\") " pod="kube-system/kas-connection-checker-685895c7c9-985h5"
Oct 30 18:36:06.930258 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.930194    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxx6s\" (UniqueName: \"kubernetes.io/projected/f16c549e-41e6-4721-ab5f-91b37f05a97c-kube-api-access-vxx6s\") pod \"kas-connection-checker-685895c7c9-v788b\" (UID: \"f16c549e-41e6-4721-ab5f-91b37f05a97c\") " pod="kube-system/kas-connection-checker-685895c7c9-v788b"
Oct 30 18:36:06.930258 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.930212    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-image-registry-private-configuration\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:06.930371 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.930271    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-trusted-ca\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:06.930371 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.930318    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-installation-pull-secrets\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:06.930431 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.930382    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:06.930464 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.930442    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-574hk\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-kube-api-access-574hk\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:06.930464 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.930460    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmgk4\" (UniqueName: \"kubernetes.io/projected/ae2cac8d-509e-4e22-b35c-ab6d662fdf24-kube-api-access-nmgk4\") pod \"kas-connection-checker-685895c7c9-tfwdw\" (UID: \"ae2cac8d-509e-4e22-b35c-ab6d662fdf24\") " pod="kube-system/kas-connection-checker-685895c7c9-tfwdw"
Oct 30 18:36:06.930523 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.930502    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-registry-certificates\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:06.930523 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.930520    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-bound-sa-token\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:06.930594 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.930535    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jms4w\" (UniqueName: \"kubernetes.io/projected/4cd65688-51e1-4b8c-8dc5-6e3771a74e66-kube-api-access-jms4w\") pod \"volume-data-source-validator-58c9f5f46d-ljxjz\" (UID: \"4cd65688-51e1-4b8c-8dc5-6e3771a74e66\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz"
Oct 30 18:36:06.945569 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.945551    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 18:36:06.945894 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.945824    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 18:36:06.946003 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.945970    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 18:36:06.946061 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.946013    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:36:06.946061 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.946013    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:36:06.955225 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.955208    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:06.955288 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.955243    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 18:36:06.955495 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.955479    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:06.955729 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.955713    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:36:06.955814 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.955749    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:36:06.955814 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.955782    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:36:06.955814 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.955785    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:36:06.957841 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.957818    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:36:06.957841 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.957823    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 18:36:06.958007 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.957843    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 18:36:06.958007 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.957855    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:36:06.958007 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.957862    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:06.958007 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.957881    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:06.958007 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.957884    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 18:36:06.958007 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.957924    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 18:36:06.958007 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.957933    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:36:06.958007 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.957961    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:06.958329 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.958312    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 18:36:06.959250 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.959238    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:06.959250 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.959246    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:06.959535 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.959518    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 18:36:06.959621 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.959536    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:06.959621 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.959569    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:06.959621 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.959585    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 18:36:06.959621 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.959599    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:06.960058 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.960043    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 18:36:06.961968 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.961952    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 18:36:06.966514 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.966490    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 18:36:06.966608 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.966547    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 18:36:06.966608 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.966573    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 18:36:06.966717 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.966632    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 18:36:06.966717 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.966704    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 18:36:06.966807 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.966714    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:36:06.966807 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.966553    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qs4k8\""
Oct 30 18:36:06.966807 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.966741    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 18:36:06.966807 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.966636    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 18:36:06.966807 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.966777    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 18:36:06.967022 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.966954    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-bxft2\""
Oct 30 18:36:06.967022 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.966883    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 18:36:06.967022 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.967000    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d6n4c\""
Oct 30 18:36:06.967189 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.967022    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 18:36:06.967189 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.967032    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 18:36:06.967189 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.967111    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 18:36:06.967189 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.967117    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 18:36:06.967392 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.967332    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:36:06.967392 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.967370    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-5rxk7\""
Oct 30 18:36:06.980849 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:06.980831    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:07.031214 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031193    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-stats-auth\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.031214 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031215    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d2b52b9e-4797-4a2e-953e-7da0f61bca87-config\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:07.031351 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031231    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97wlp\" (UniqueName: \"kubernetes.io/projected/d2b52b9e-4797-4a2e-953e-7da0f61bca87-kube-api-access-97wlp\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:07.031351 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031254    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd89g\" (UniqueName: \"kubernetes.io/projected/b0939980-01bb-4ae3-9c95-5e190b719cbe-kube-api-access-jd89g\") pod \"kas-connection-checker-685895c7c9-985h5\" (UID: \"b0939980-01bb-4ae3-9c95-5e190b719cbe\") " pod="kube-system/kas-connection-checker-685895c7c9-985h5"
Oct 30 18:36:07.031351 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031338    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.031461 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031449    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0ba138d-0a24-4f77-b1f4-c0861a48797f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:07.031493 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031478    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:07.031551 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031495    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:07.031551 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031533    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9mdl\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-kube-api-access-p9mdl\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.031661 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031586    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-trusted-ca\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.031661 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031619    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd26f706-5971-4c47-a63e-699e7962ef8e-ca-trust-extracted\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.031661 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031646    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-bound-sa-token\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.031765 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031673    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b90705d-6abb-481d-8af0-89e542709329-serving-cert\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.031765 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031717    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b90705d-6abb-481d-8af0-89e542709329-tmp\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.031765 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031735    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734db718-c901-4fe7-a8e1-b90823ccd66c-tmp-dir\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:07.031765 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031752    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5cf855c4-4e7e-4669-9adc-583e22e27e73-config\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.031880 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031788    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.031880 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031813    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl4tg\" (UniqueName: \"kubernetes.io/projected/5cf855c4-4e7e-4669-9adc-583e22e27e73-kube-api-access-tl4tg\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.031880 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031842    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-bound-sa-token\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.031983 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031896    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90705d-6abb-481d-8af0-89e542709329-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.031983 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031938    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:07.031983 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031971    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9mtz\" (UniqueName: \"kubernetes.io/projected/2c77e172-a80d-4eda-986d-ec8c3c52e131-kube-api-access-f9mtz\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.032157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.031988    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-image-registry-private-configuration\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.032157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032005    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4bee5094-7886-4fa9-aa7c-0ddc3d518619-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:07.032157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032047    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90705d-6abb-481d-8af0-89e542709329-service-ca-bundle\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.032157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032071    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b52b9e-4797-4a2e-953e-7da0f61bca87-serving-cert\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:07.032157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032112    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e32eb376-a16f-421c-8958-a410a5650964-ca-trust-extracted\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.032157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032148    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5cf855c4-4e7e-4669-9adc-583e22e27e73-serving-cert\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.032416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032183    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxx6s\" (UniqueName: \"kubernetes.io/projected/f16c549e-41e6-4721-ab5f-91b37f05a97c-kube-api-access-vxx6s\") pod \"kas-connection-checker-685895c7c9-v788b\" (UID: \"f16c549e-41e6-4721-ab5f-91b37f05a97c\") " pod="kube-system/kas-connection-checker-685895c7c9-v788b"
Oct 30 18:36:07.032416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032214    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-image-registry-private-configuration\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.032416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032241    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-installation-pull-secrets\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.032416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032266    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cf855c4-4e7e-4669-9adc-583e22e27e73-trusted-ca\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.032416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032294    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:07.032416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032397    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-trusted-ca\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.032416 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032402    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e32eb376-a16f-421c-8958-a410a5650964-ca-trust-extracted\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.032680 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032439    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.032680 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032467    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkr9h\" (UniqueName: \"kubernetes.io/projected/272efc3d-7980-4fd1-a20d-9bd023f48bec-kube-api-access-dkr9h\") pod \"ingress-canary-z2nsf\" (UID: \"272efc3d-7980-4fd1-a20d-9bd023f48bec\") " pod="openshift-ingress-canary/ingress-canary-z2nsf"
Oct 30 18:36:07.032680 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032493    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-certificates\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.032680 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.032511    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:07.032680 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032519    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84gss\" (UniqueName: \"kubernetes.io/projected/4bee5094-7886-4fa9-aa7c-0ddc3d518619-kube-api-access-84gss\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:07.032680 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.032528    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6684c84797-p6964: secret "image-registry-tls" not found
Oct 30 18:36:07.032680 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.032567    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls podName:e32eb376-a16f-421c-8958-a410a5650964 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:07.532553225 +0000 UTC m=+30.826619739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls") pod "image-registry-6684c84797-p6964" (UID: "e32eb376-a16f-421c-8958-a410a5650964") : secret "image-registry-tls" not found
Oct 30 18:36:07.032680 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032609    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032670    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dn5r\" (UniqueName: \"kubernetes.io/projected/0b90705d-6abb-481d-8af0-89e542709329-kube-api-access-7dn5r\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032725    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/272efc3d-7980-4fd1-a20d-9bd023f48bec-cert\") pod \"ingress-canary-z2nsf\" (UID: \"272efc3d-7980-4fd1-a20d-9bd023f48bec\") " pod="openshift-ingress-canary/ingress-canary-z2nsf"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032754    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgfnf\" (UniqueName: \"kubernetes.io/projected/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-kube-api-access-pgfnf\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032782    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-trusted-ca\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032805    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4bee5094-7886-4fa9-aa7c-0ddc3d518619-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032836    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-574hk\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-kube-api-access-574hk\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032866    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-installation-pull-secrets\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032895    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-default-certificate\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032916    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmgk4\" (UniqueName: \"kubernetes.io/projected/ae2cac8d-509e-4e22-b35c-ab6d662fdf24-kube-api-access-nmgk4\") pod \"kas-connection-checker-685895c7c9-tfwdw\" (UID: \"ae2cac8d-509e-4e22-b35c-ab6d662fdf24\") " pod="kube-system/kas-connection-checker-685895c7c9-tfwdw"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032934    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f478aac2-de91-4b13-836b-ba09809222f0-nginx-conf\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032963    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0b90705d-6abb-481d-8af0-89e542709329-snapshots\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.032988    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9q7l\" (UniqueName: \"kubernetes.io/projected/734db718-c901-4fe7-a8e1-b90823ccd66c-kube-api-access-s9q7l\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:07.033024 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.033030    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-registry-certificates\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.033572 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.033067    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jms4w\" (UniqueName: \"kubernetes.io/projected/4cd65688-51e1-4b8c-8dc5-6e3771a74e66-kube-api-access-jms4w\") pod \"volume-data-source-validator-58c9f5f46d-ljxjz\" (UID: \"4cd65688-51e1-4b8c-8dc5-6e3771a74e66\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz"
Oct 30 18:36:07.033572 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.033168    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/734db718-c901-4fe7-a8e1-b90823ccd66c-config-volume\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:07.033572 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.033196    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.033572 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.033229    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5vxf\" (UniqueName: \"kubernetes.io/projected/f0ba138d-0a24-4f77-b1f4-c0861a48797f-kube-api-access-t5vxf\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:07.033572 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.033263    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnnxf\" (UniqueName: \"kubernetes.io/projected/9e5ac899-ecba-4c42-aa7a-e34e14269efa-kube-api-access-lnnxf\") pod \"network-check-source-6f78d59f99-hjr2n\" (UID: \"9e5ac899-ecba-4c42-aa7a-e34e14269efa\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n"
Oct 30 18:36:07.033572 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.033503    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-registry-certificates\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.036002 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.035983    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-installation-pull-secrets\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.036073 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.036000    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-image-registry-private-configuration\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.045175 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.045132    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd89g\" (UniqueName: \"kubernetes.io/projected/b0939980-01bb-4ae3-9c95-5e190b719cbe-kube-api-access-jd89g\") pod \"kas-connection-checker-685895c7c9-985h5\" (UID: \"b0939980-01bb-4ae3-9c95-5e190b719cbe\") " pod="kube-system/kas-connection-checker-685895c7c9-985h5"
Oct 30 18:36:07.047832 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.047808    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-bound-sa-token\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.048238 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.048220    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-574hk\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-kube-api-access-574hk\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.048543 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.048521    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmgk4\" (UniqueName: \"kubernetes.io/projected/ae2cac8d-509e-4e22-b35c-ab6d662fdf24-kube-api-access-nmgk4\") pod \"kas-connection-checker-685895c7c9-tfwdw\" (UID: \"ae2cac8d-509e-4e22-b35c-ab6d662fdf24\") " pod="kube-system/kas-connection-checker-685895c7c9-tfwdw"
Oct 30 18:36:07.048997 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.048975    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxx6s\" (UniqueName: \"kubernetes.io/projected/f16c549e-41e6-4721-ab5f-91b37f05a97c-kube-api-access-vxx6s\") pod \"kas-connection-checker-685895c7c9-v788b\" (UID: \"f16c549e-41e6-4721-ab5f-91b37f05a97c\") " pod="kube-system/kas-connection-checker-685895c7c9-v788b"
Oct 30 18:36:07.049225 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.049208    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jms4w\" (UniqueName: \"kubernetes.io/projected/4cd65688-51e1-4b8c-8dc5-6e3771a74e66-kube-api-access-jms4w\") pod \"volume-data-source-validator-58c9f5f46d-ljxjz\" (UID: \"4cd65688-51e1-4b8c-8dc5-6e3771a74e66\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz"
Oct 30 18:36:07.134177 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134152    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnnxf\" (UniqueName: \"kubernetes.io/projected/9e5ac899-ecba-4c42-aa7a-e34e14269efa-kube-api-access-lnnxf\") pod \"network-check-source-6f78d59f99-hjr2n\" (UID: \"9e5ac899-ecba-4c42-aa7a-e34e14269efa\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n"
Oct 30 18:36:07.134262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134179    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-stats-auth\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.134262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134197    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d2b52b9e-4797-4a2e-953e-7da0f61bca87-config\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:07.134262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134212    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97wlp\" (UniqueName: \"kubernetes.io/projected/d2b52b9e-4797-4a2e-953e-7da0f61bca87-kube-api-access-97wlp\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:07.134262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134229    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.134262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134245    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0ba138d-0a24-4f77-b1f4-c0861a48797f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:07.134262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134262    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134285    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134311    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mdl\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-kube-api-access-p9mdl\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134337    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd26f706-5971-4c47-a63e-699e7962ef8e-ca-trust-extracted\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.134344    2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134361    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-bound-sa-token\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134387    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b90705d-6abb-481d-8af0-89e542709329-serving-cert\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.134405    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.134417    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:07.6343908 +0000 UTC m=+30.928457320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : secret "router-metrics-certs-default" not found
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134446    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b90705d-6abb-481d-8af0-89e542709329-tmp\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.134457    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls podName:9e9032e7-1ec1-4a5d-8e28-622d4fd01392 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:07.634442084 +0000 UTC m=+30.928508601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-vzrmj" (UID: "9e9032e7-1ec1-4a5d-8e28-622d4fd01392") : secret "samples-operator-tls" not found
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134486    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734db718-c901-4fe7-a8e1-b90823ccd66c-tmp-dir\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134515    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5cf855c4-4e7e-4669-9adc-583e22e27e73-config\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.134553 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134543    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134572    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl4tg\" (UniqueName: \"kubernetes.io/projected/5cf855c4-4e7e-4669-9adc-583e22e27e73-kube-api-access-tl4tg\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134615    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90705d-6abb-481d-8af0-89e542709329-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134644    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134673    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9mtz\" (UniqueName: \"kubernetes.io/projected/2c77e172-a80d-4eda-986d-ec8c3c52e131-kube-api-access-f9mtz\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134697    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-image-registry-private-configuration\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134720    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4bee5094-7886-4fa9-aa7c-0ddc3d518619-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134747    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90705d-6abb-481d-8af0-89e542709329-service-ca-bundle\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134769    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b52b9e-4797-4a2e-953e-7da0f61bca87-serving-cert\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134794    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5cf855c4-4e7e-4669-9adc-583e22e27e73-serving-cert\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134828    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cf855c4-4e7e-4669-9adc-583e22e27e73-trusted-ca\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134839    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd26f706-5971-4c47-a63e-699e7962ef8e-ca-trust-extracted\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134843    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b90705d-6abb-481d-8af0-89e542709329-tmp\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134853    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134875    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d2b52b9e-4797-4a2e-953e-7da0f61bca87-config\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134920    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkr9h\" (UniqueName: \"kubernetes.io/projected/272efc3d-7980-4fd1-a20d-9bd023f48bec-kube-api-access-dkr9h\") pod \"ingress-canary-z2nsf\" (UID: \"272efc3d-7980-4fd1-a20d-9bd023f48bec\") " pod="openshift-ingress-canary/ingress-canary-z2nsf"
Oct 30 18:36:07.135157 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134946    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-certificates\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.134971    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84gss\" (UniqueName: \"kubernetes.io/projected/4bee5094-7886-4fa9-aa7c-0ddc3d518619-kube-api-access-84gss\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135024    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135052    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dn5r\" (UniqueName: \"kubernetes.io/projected/0b90705d-6abb-481d-8af0-89e542709329-kube-api-access-7dn5r\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135076    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/272efc3d-7980-4fd1-a20d-9bd023f48bec-cert\") pod \"ingress-canary-z2nsf\" (UID: \"272efc3d-7980-4fd1-a20d-9bd023f48bec\") " pod="openshift-ingress-canary/ingress-canary-z2nsf"
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135117    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgfnf\" (UniqueName: \"kubernetes.io/projected/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-kube-api-access-pgfnf\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135143    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-trusted-ca\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.135170    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.135227    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls podName:f0ba138d-0a24-4f77-b1f4-c0861a48797f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:07.635210116 +0000 UTC m=+30.929276641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n92pf" (UID: "f0ba138d-0a24-4f77-b1f4-c0861a48797f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.135247    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:07.635237646 +0000 UTC m=+30.929304146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : configmap references non-existent config key: service-ca.crt
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135266    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4bee5094-7886-4fa9-aa7c-0ddc3d518619-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.135294    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135302    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-installation-pull-secrets\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135331    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-default-certificate\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135367    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f478aac2-de91-4b13-836b-ba09809222f0-nginx-conf\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:07.136111 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135394    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0b90705d-6abb-481d-8af0-89e542709329-snapshots\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.136918 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135417    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9q7l\" (UniqueName: \"kubernetes.io/projected/734db718-c901-4fe7-a8e1-b90823ccd66c-kube-api-access-s9q7l\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:07.136918 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135453    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/734db718-c901-4fe7-a8e1-b90823ccd66c-config-volume\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:07.136918 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135477    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.136918 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135507    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5vxf\" (UniqueName: \"kubernetes.io/projected/f0ba138d-0a24-4f77-b1f4-c0861a48797f-kube-api-access-t5vxf\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:07.136918 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.135623    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734db718-c901-4fe7-a8e1-b90823ccd66c-tmp-dir\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:07.136918 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.136162    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-trusted-ca\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.136918 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.136172    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90705d-6abb-481d-8af0-89e542709329-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.136918 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.136400    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4bee5094-7886-4fa9-aa7c-0ddc3d518619-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:07.136918 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.136399    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5cf855c4-4e7e-4669-9adc-583e22e27e73-config\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.137354 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.136921    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90705d-6abb-481d-8af0-89e542709329-service-ca-bundle\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.137354 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.136962    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls podName:734db718-c901-4fe7-a8e1-b90823ccd66c nodeName:}" failed. No retries permitted until 2026-10-30 18:36:07.636947069 +0000 UTC m=+30.931013568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls") pod "dns-default-jcn7p" (UID: "734db718-c901-4fe7-a8e1-b90823ccd66c") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:07.137354 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.137016    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b90705d-6abb-481d-8af0-89e542709329-serving-cert\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.137354 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.137137    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:07.137561 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.137471    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-certificates\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.138139 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.137617    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f478aac2-de91-4b13-836b-ba09809222f0-nginx-conf\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:07.138139 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.137678    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-image-registry-private-configuration\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.138139 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.137739    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:36:07.138139 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.137768    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:07.138139 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.137781    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f45597c6-hfv8h: secret "image-registry-tls" not found
Oct 30 18:36:07.138139 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.137784    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert podName:f478aac2-de91-4b13-836b-ba09809222f0 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:07.637769862 +0000 UTC m=+30.931836397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k9dgt" (UID: "f478aac2-de91-4b13-836b-ba09809222f0") : secret "networking-console-plugin-cert" not found
Oct 30 18:36:07.138139 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.137780    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0ba138d-0a24-4f77-b1f4-c0861a48797f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:07.138139 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.137829    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls podName:cd26f706-5971-4c47-a63e-699e7962ef8e nodeName:}" failed. No retries permitted until 2026-10-30 18:36:07.637814512 +0000 UTC m=+30.931881031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls") pod "image-registry-76f45597c6-hfv8h" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e") : secret "image-registry-tls" not found
Oct 30 18:36:07.138139 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.137913    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0b90705d-6abb-481d-8af0-89e542709329-snapshots\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.138139 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.138034    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cf855c4-4e7e-4669-9adc-583e22e27e73-trusted-ca\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.138854 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.138200    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/734db718-c901-4fe7-a8e1-b90823ccd66c-config-volume\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:07.139560 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.139534    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-default-certificate\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.139965 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.139943    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b52b9e-4797-4a2e-953e-7da0f61bca87-serving-cert\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:07.140385 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.140364    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4bee5094-7886-4fa9-aa7c-0ddc3d518619-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:07.140749 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.140732    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-installation-pull-secrets\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.140941 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.140923    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-stats-auth\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.141005 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.140971    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/272efc3d-7980-4fd1-a20d-9bd023f48bec-cert\") pod \"ingress-canary-z2nsf\" (UID: \"272efc3d-7980-4fd1-a20d-9bd023f48bec\") " pod="openshift-ingress-canary/ingress-canary-z2nsf"
Oct 30 18:36:07.141144 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.141130    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5cf855c4-4e7e-4669-9adc-583e22e27e73-serving-cert\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.150142 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.150119    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnnxf\" (UniqueName: \"kubernetes.io/projected/9e5ac899-ecba-4c42-aa7a-e34e14269efa-kube-api-access-lnnxf\") pod \"network-check-source-6f78d59f99-hjr2n\" (UID: \"9e5ac899-ecba-4c42-aa7a-e34e14269efa\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n"
Oct 30 18:36:07.150635 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.150614    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97wlp\" (UniqueName: \"kubernetes.io/projected/d2b52b9e-4797-4a2e-953e-7da0f61bca87-kube-api-access-97wlp\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:07.152563 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.152535    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dn5r\" (UniqueName: \"kubernetes.io/projected/0b90705d-6abb-481d-8af0-89e542709329-kube-api-access-7dn5r\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.153571 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.153544    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgfnf\" (UniqueName: \"kubernetes.io/projected/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-kube-api-access-pgfnf\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:07.153671 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.153550    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9mdl\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-kube-api-access-p9mdl\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.153813 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.153753    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl4tg\" (UniqueName: \"kubernetes.io/projected/5cf855c4-4e7e-4669-9adc-583e22e27e73-kube-api-access-tl4tg\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.153813 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.153772    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkr9h\" (UniqueName: \"kubernetes.io/projected/272efc3d-7980-4fd1-a20d-9bd023f48bec-kube-api-access-dkr9h\") pod \"ingress-canary-z2nsf\" (UID: \"272efc3d-7980-4fd1-a20d-9bd023f48bec\") " pod="openshift-ingress-canary/ingress-canary-z2nsf"
Oct 30 18:36:07.153965 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.153871    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5vxf\" (UniqueName: \"kubernetes.io/projected/f0ba138d-0a24-4f77-b1f4-c0861a48797f-kube-api-access-t5vxf\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:07.154067 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.154049    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84gss\" (UniqueName: \"kubernetes.io/projected/4bee5094-7886-4fa9-aa7c-0ddc3d518619-kube-api-access-84gss\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:07.154172 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.154155    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9q7l\" (UniqueName: \"kubernetes.io/projected/734db718-c901-4fe7-a8e1-b90823ccd66c-kube-api-access-s9q7l\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:07.155327 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.155311    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-bound-sa-token\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.155540 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.155525    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9mtz\" (UniqueName: \"kubernetes.io/projected/2c77e172-a80d-4eda-986d-ec8c3c52e131-kube-api-access-f9mtz\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.159663 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.159649    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-985h5"
Oct 30 18:36:07.178956 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.178937    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-tfwdw"
Oct 30 18:36:07.193194 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.193162    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz"
Oct 30 18:36:07.205872 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.201385    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-v788b"
Oct 30 18:36:07.208579 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.208509    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:07.215249 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.215111    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:07.228239 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.226405    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:07.253809 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.229903    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:07.253809 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.231553    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:07.253809 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.252459    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:07.279516 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.261113    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:36:07.279516 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.261592    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n"
Oct 30 18:36:07.279516 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.261665    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dnnzq\""
Oct 30 18:36:07.279516 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.274988    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z2nsf"
Oct 30 18:36:07.451844 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.451811    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-985h5"]
Oct 30 18:36:07.454691 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:36:07.454667    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0939980_01bb_4ae3_9c95_5e190b719cbe.slice/crio-eb46efb03bca9a23e02ad293848408bc0d72301fe54b8942d15b471d323bc9b5 WatchSource:0}: Error finding container eb46efb03bca9a23e02ad293848408bc0d72301fe54b8942d15b471d323bc9b5: Status 404 returned error can't find the container with id eb46efb03bca9a23e02ad293848408bc0d72301fe54b8942d15b471d323bc9b5
Oct 30 18:36:07.473951 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.473923    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-tfwdw"]
Oct 30 18:36:07.478652 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:36:07.478622    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2cac8d_509e_4e22_b35c_ab6d662fdf24.slice/crio-abd8ee6926056d9e202f1f341f371a7d577561fc4fc8005f5652ff22ac4fb453 WatchSource:0}: Error finding container abd8ee6926056d9e202f1f341f371a7d577561fc4fc8005f5652ff22ac4fb453: Status 404 returned error can't find the container with id abd8ee6926056d9e202f1f341f371a7d577561fc4fc8005f5652ff22ac4fb453
Oct 30 18:36:07.496123 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.496040    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-985h5" event={"ID":"b0939980-01bb-4ae3-9c95-5e190b719cbe","Type":"ContainerStarted","Data":"eb46efb03bca9a23e02ad293848408bc0d72301fe54b8942d15b471d323bc9b5"}
Oct 30 18:36:07.498088 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.498064    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-tfwdw" event={"ID":"ae2cac8d-509e-4e22-b35c-ab6d662fdf24","Type":"ContainerStarted","Data":"abd8ee6926056d9e202f1f341f371a7d577561fc4fc8005f5652ff22ac4fb453"}
Oct 30 18:36:07.512503 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.512481    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-v788b"]
Oct 30 18:36:07.518607 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:36:07.518580    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16c549e_41e6_4721_ab5f_91b37f05a97c.slice/crio-5a42aedc4ebe3ce8adbed1030f1b5994ef0e8cc402352acf984b72350f25a3b1 WatchSource:0}: Error finding container 5a42aedc4ebe3ce8adbed1030f1b5994ef0e8cc402352acf984b72350f25a3b1: Status 404 returned error can't find the container with id 5a42aedc4ebe3ce8adbed1030f1b5994ef0e8cc402352acf984b72350f25a3b1
Oct 30 18:36:07.542944 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.542336    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:07.542944 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.542556    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:07.542944 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.542573    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6684c84797-p6964: secret "image-registry-tls" not found
Oct 30 18:36:07.542944 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.542629    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls podName:e32eb376-a16f-421c-8958-a410a5650964 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:08.542610049 +0000 UTC m=+31.836676568 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls") pod "image-registry-6684c84797-p6964" (UID: "e32eb376-a16f-421c-8958-a410a5650964") : secret "image-registry-tls" not found
Oct 30 18:36:07.560717 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.560654    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"]
Oct 30 18:36:07.580225 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.580146    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz"]
Oct 30 18:36:07.582413 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.582391    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"]
Oct 30 18:36:07.592033 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.592007    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z2nsf"]
Oct 30 18:36:07.592929 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:36:07.592815    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod272efc3d_7980_4fd1_a20d_9bd023f48bec.slice/crio-c4f431637bd8ea72fd98b092b53df10b00f0d0a3ba3a856b957e0501a506bece WatchSource:0}: Error finding container c4f431637bd8ea72fd98b092b53df10b00f0d0a3ba3a856b957e0501a506bece: Status 404 returned error can't find the container with id c4f431637bd8ea72fd98b092b53df10b00f0d0a3ba3a856b957e0501a506bece
Oct 30 18:36:07.594611 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:36:07.594578    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bee5094_7886_4fa9_aa7c_0ddc3d518619.slice/crio-518b6b63e2b339f44b2facd01785314e1b0649197ae8ea149933195c63c6821c WatchSource:0}: Error finding container 518b6b63e2b339f44b2facd01785314e1b0649197ae8ea149933195c63c6821c: Status 404 returned error can't find the container with id 518b6b63e2b339f44b2facd01785314e1b0649197ae8ea149933195c63c6821c
Oct 30 18:36:07.611720 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.611690    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-qhk9m"]
Oct 30 18:36:07.612923 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.612888    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-68flm"]
Oct 30 18:36:07.613684 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:36:07.613663    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b90705d_6abb_481d_8af0_89e542709329.slice/crio-6ab635073ce18d9860646ba15f6fd95ea3ec30b03f52d9724794589a435d01b9 WatchSource:0}: Error finding container 6ab635073ce18d9860646ba15f6fd95ea3ec30b03f52d9724794589a435d01b9: Status 404 returned error can't find the container with id 6ab635073ce18d9860646ba15f6fd95ea3ec30b03f52d9724794589a435d01b9
Oct 30 18:36:07.616902 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:36:07.616863    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cf855c4_4e7e_4669_9adc_583e22e27e73.slice/crio-8ebf01bb27e75d8f710025340685c56c55877a45e266253db115f25358a5b0de WatchSource:0}: Error finding container 8ebf01bb27e75d8f710025340685c56c55877a45e266253db115f25358a5b0de: Status 404 returned error can't find the container with id 8ebf01bb27e75d8f710025340685c56c55877a45e266253db115f25358a5b0de
Oct 30 18:36:07.637510 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.637488    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n"]
Oct 30 18:36:07.640445 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:36:07.640426    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e5ac899_ecba_4c42_aa7a_e34e14269efa.slice/crio-d34814233153f704c3a04db0cd80b7994903b3e1dc582de690be12869ca63296 WatchSource:0}: Error finding container d34814233153f704c3a04db0cd80b7994903b3e1dc582de690be12869ca63296: Status 404 returned error can't find the container with id d34814233153f704c3a04db0cd80b7994903b3e1dc582de690be12869ca63296
Oct 30 18:36:07.642758 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.642739    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:07.642824 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.642768    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:07.642824 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.642795    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.642890 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.642836    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:07.642890 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.642876    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:07.642943 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.642907    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:07.642943 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.642930    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:07.643034 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643023    2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:36:07.643066 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643061    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:08.643049835 +0000 UTC m=+31.937116336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : secret "router-metrics-certs-default" not found
Oct 30 18:36:07.643128 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643123    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:07.643159 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643145    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls podName:734db718-c901-4fe7-a8e1-b90823ccd66c nodeName:}" failed. No retries permitted until 2026-10-30 18:36:08.643137906 +0000 UTC m=+31.937204405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls") pod "dns-default-jcn7p" (UID: "734db718-c901-4fe7-a8e1-b90823ccd66c") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:07.643197 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643179    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:36:07.643240 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643197    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls podName:9e9032e7-1ec1-4a5d-8e28-622d4fd01392 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:08.643191543 +0000 UTC m=+31.937258043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-vzrmj" (UID: "9e9032e7-1ec1-4a5d-8e28-622d4fd01392") : secret "samples-operator-tls" not found
Oct 30 18:36:07.643302 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643289    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:08.643276839 +0000 UTC m=+31.937343354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : configmap references non-existent config key: service-ca.crt
Oct 30 18:36:07.643339 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643292    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:36:07.643368 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643327    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:07.643395 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643332    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert podName:f478aac2-de91-4b13-836b-ba09809222f0 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:08.643323888 +0000 UTC m=+31.937390388 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k9dgt" (UID: "f478aac2-de91-4b13-836b-ba09809222f0") : secret "networking-console-plugin-cert" not found
Oct 30 18:36:07.643429 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643393    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:07.643429 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643407    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f45597c6-hfv8h: secret "image-registry-tls" not found
Oct 30 18:36:07.643429 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643421    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls podName:f0ba138d-0a24-4f77-b1f4-c0861a48797f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:08.643393451 +0000 UTC m=+31.937459951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n92pf" (UID: "f0ba138d-0a24-4f77-b1f4-c0861a48797f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:07.643519 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:07.643437    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls podName:cd26f706-5971-4c47-a63e-699e7962ef8e nodeName:}" failed. No retries permitted until 2026-10-30 18:36:08.643428188 +0000 UTC m=+31.937494690 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls") pod "image-registry-76f45597c6-hfv8h" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e") : secret "image-registry-tls" not found
Oct 30 18:36:07.961741 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:07.961710    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:08.135980 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.135949    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:08.248379 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.248316    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:08.248379 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.248344    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:08.253221 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.253171    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vgk4h\""
Oct 30 18:36:08.253221 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.253179    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-phzbf\""
Oct 30 18:36:08.253376 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.253287    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:36:08.503663 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.503537    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z2nsf" event={"ID":"272efc3d-7980-4fd1-a20d-9bd023f48bec","Type":"ContainerStarted","Data":"c4f431637bd8ea72fd98b092b53df10b00f0d0a3ba3a856b957e0501a506bece"}
Oct 30 18:36:08.505452 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.505386    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-qhk9m" event={"ID":"0b90705d-6abb-481d-8af0-89e542709329","Type":"ContainerStarted","Data":"6ab635073ce18d9860646ba15f6fd95ea3ec30b03f52d9724794589a435d01b9"}
Oct 30 18:36:08.507030 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.506969    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc" event={"ID":"4bee5094-7886-4fa9-aa7c-0ddc3d518619","Type":"ContainerStarted","Data":"518b6b63e2b339f44b2facd01785314e1b0649197ae8ea149933195c63c6821c"}
Oct 30 18:36:08.510217 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.510187    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-985h5" event={"ID":"b0939980-01bb-4ae3-9c95-5e190b719cbe","Type":"ContainerStarted","Data":"71989cedc9624056b60ea6cb06bf14aff400e403ee411fdc1ab27711de73a075"}
Oct 30 18:36:08.511741 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.511715    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n" event={"ID":"9e5ac899-ecba-4c42-aa7a-e34e14269efa","Type":"ContainerStarted","Data":"d34814233153f704c3a04db0cd80b7994903b3e1dc582de690be12869ca63296"}
Oct 30 18:36:08.516351 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.516271    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm" event={"ID":"5cf855c4-4e7e-4669-9adc-583e22e27e73","Type":"ContainerStarted","Data":"8ebf01bb27e75d8f710025340685c56c55877a45e266253db115f25358a5b0de"}
Oct 30 18:36:08.520198 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.519684    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-v788b" event={"ID":"f16c549e-41e6-4721-ab5f-91b37f05a97c","Type":"ContainerStarted","Data":"744fbfcf421cd948310d82672f8b9121f72501b7b15093238b46427d2a0d7267"}
Oct 30 18:36:08.520198 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.519724    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-v788b" event={"ID":"f16c549e-41e6-4721-ab5f-91b37f05a97c","Type":"ContainerStarted","Data":"5a42aedc4ebe3ce8adbed1030f1b5994ef0e8cc402352acf984b72350f25a3b1"}
Oct 30 18:36:08.522743 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.522689    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-tfwdw" event={"ID":"ae2cac8d-509e-4e22-b35c-ab6d662fdf24","Type":"ContainerStarted","Data":"304415e8a91ebedfe61e58599d7d49669a66e3b3c41cbba9ac763e431863579e"}
Oct 30 18:36:08.524613 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.524554    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg" event={"ID":"d2b52b9e-4797-4a2e-953e-7da0f61bca87","Type":"ContainerStarted","Data":"f35a1bc16f4abc1d8257b702bc5da0459be08a98e1f4ff005bd0693db0140d57"}
Oct 30 18:36:08.526812 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.525973    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz" event={"ID":"4cd65688-51e1-4b8c-8dc5-6e3771a74e66","Type":"ContainerStarted","Data":"5886c3667fcb917c58b17d5a153971827aea2790497f54c1a53f6d71ad31dd84"}
Oct 30 18:36:08.526947 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.526902    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-985h5" podStartSLOduration=233.526887762 podStartE2EDuration="3m53.526887762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:08.526660067 +0000 UTC m=+31.820726599" watchObservedRunningTime="2026-10-30 18:36:08.526887762 +0000 UTC m=+31.820954280"
Oct 30 18:36:08.551319 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.550680    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:08.551319 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.550941    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:08.551319 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.550957    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6684c84797-p6964: secret "image-registry-tls" not found
Oct 30 18:36:08.551319 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.551009    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls podName:e32eb376-a16f-421c-8958-a410a5650964 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.550992908 +0000 UTC m=+33.845059423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls") pod "image-registry-6684c84797-p6964" (UID: "e32eb376-a16f-421c-8958-a410a5650964") : secret "image-registry-tls" not found
Oct 30 18:36:08.566309 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.564877    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-tfwdw" podStartSLOduration=233.564861843 podStartE2EDuration="3m53.564861843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:08.545475585 +0000 UTC m=+31.839542107" watchObservedRunningTime="2026-10-30 18:36:08.564861843 +0000 UTC m=+31.858928366"
Oct 30 18:36:08.566309 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.565580    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-v788b" podStartSLOduration=233.56556709 podStartE2EDuration="3m53.56556709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:08.564268001 +0000 UTC m=+31.858334549" watchObservedRunningTime="2026-10-30 18:36:08.56556709 +0000 UTC m=+31.859633613"
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.651853    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.651970    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.652038    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.652125    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.652183    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.652211    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:08.652239    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.652430    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.652496    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls podName:9e9032e7-1ec1-4a5d-8e28-622d4fd01392 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.652479673 +0000 UTC m=+33.946546187 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-vzrmj" (UID: "9e9032e7-1ec1-4a5d-8e28-622d4fd01392") : secret "samples-operator-tls" not found
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.652934    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.652917876 +0000 UTC m=+33.946984390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : configmap references non-existent config key: service-ca.crt
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.653835    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.653882    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls podName:f0ba138d-0a24-4f77-b1f4-c0861a48797f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.653868588 +0000 UTC m=+33.947935102 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n92pf" (UID: "f0ba138d-0a24-4f77-b1f4-c0861a48797f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.653936    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.653972    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert podName:f478aac2-de91-4b13-836b-ba09809222f0 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.653960295 +0000 UTC m=+33.948026811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k9dgt" (UID: "f478aac2-de91-4b13-836b-ba09809222f0") : secret "networking-console-plugin-cert" not found
Oct 30 18:36:08.655262 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.654294    2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:36:08.656158 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.654339    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.654326228 +0000 UTC m=+33.948392743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : secret "router-metrics-certs-default" not found
Oct 30 18:36:08.656158 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.654403    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:08.656158 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.654415    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f45597c6-hfv8h: secret "image-registry-tls" not found
Oct 30 18:36:08.656158 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.654445    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls podName:cd26f706-5971-4c47-a63e-699e7962ef8e nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.654435883 +0000 UTC m=+33.948502400 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls") pod "image-registry-76f45597c6-hfv8h" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e") : secret "image-registry-tls" not found
Oct 30 18:36:08.656158 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.655133    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:08.656158 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:08.655227    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls podName:734db718-c901-4fe7-a8e1-b90823ccd66c nodeName:}" failed. No retries permitted until 2026-10-30 18:36:10.655160526 +0000 UTC m=+33.949227041 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls") pod "dns-default-jcn7p" (UID: "734db718-c901-4fe7-a8e1-b90823ccd66c") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:10.537166 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.536414    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz" event={"ID":"4cd65688-51e1-4b8c-8dc5-6e3771a74e66","Type":"ContainerStarted","Data":"0a39fd9f256a58de30faf4efc01420e96858e1df219eda21b8c1eb25605b9e8a"}
Oct 30 18:36:10.555196 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.554923    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz" podStartSLOduration=203.279344095 podStartE2EDuration="3m25.554905765s" totalImagesPullingTime="2.27556167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:10.552986474 +0000 UTC m=+33.847053022" watchObservedRunningTime="2026-10-30 18:36:10.554905765 +0000 UTC m=+33.848972290"
Oct 30 18:36:10.573346 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.573312    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:10.573609 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.573584    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:10.573609 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.573608    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6684c84797-p6964: secret "image-registry-tls" not found
Oct 30 18:36:10.573764 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.573659    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls podName:e32eb376-a16f-421c-8958-a410a5650964 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:14.573644127 +0000 UTC m=+37.867710641 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls") pod "image-registry-6684c84797-p6964" (UID: "e32eb376-a16f-421c-8958-a410a5650964") : secret "image-registry-tls" not found
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.674083    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.674157    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.674189    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.674229    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.674264    2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.674299    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.674326    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:14.674309982 +0000 UTC m=+37.968376488 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : secret "router-metrics-certs-default" not found
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.674369    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.674385    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.674403    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls podName:f0ba138d-0a24-4f77-b1f4-c0861a48797f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:14.674392382 +0000 UTC m=+37.968458895 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n92pf" (UID: "f0ba138d-0a24-4f77-b1f4-c0861a48797f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.674441    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.674471    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert podName:f478aac2-de91-4b13-836b-ba09809222f0 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:14.674461578 +0000 UTC m=+37.968528081 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k9dgt" (UID: "f478aac2-de91-4b13-836b-ba09809222f0") : secret "networking-console-plugin-cert" not found
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.674487    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.674560    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.674570    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f45597c6-hfv8h: secret "image-registry-tls" not found
Oct 30 18:36:10.675299 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.674594    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls podName:cd26f706-5971-4c47-a63e-699e7962ef8e nodeName:}" failed. No retries permitted until 2026-10-30 18:36:14.674586141 +0000 UTC m=+37.968652642 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls") pod "image-registry-76f45597c6-hfv8h" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e") : secret "image-registry-tls" not found
Oct 30 18:36:10.676180 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.675009    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:36:10.676180 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.675065    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls podName:9e9032e7-1ec1-4a5d-8e28-622d4fd01392 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:14.675049169 +0000 UTC m=+37.969115686 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-vzrmj" (UID: "9e9032e7-1ec1-4a5d-8e28-622d4fd01392") : secret "samples-operator-tls" not found
Oct 30 18:36:10.676180 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.675108    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:14.675072268 +0000 UTC m=+37.969138789 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : configmap references non-existent config key: service-ca.crt
Oct 30 18:36:10.676180 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.675184    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:10.676180 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.675232    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls podName:734db718-c901-4fe7-a8e1-b90823ccd66c nodeName:}" failed. No retries permitted until 2026-10-30 18:36:14.675217604 +0000 UTC m=+37.969284119 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls") pod "dns-default-jcn7p" (UID: "734db718-c901-4fe7-a8e1-b90823ccd66c") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:10.755819 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.755731    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:10.876125 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.875998    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:10.881372 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.881337    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:10.942675 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.942635    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:10.977627 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:10.977215    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:10.977627 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.977356    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:36:10.977627 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:10.977415    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs podName:74a6aabd-b72d-422d-ac40-7d07a9e56a41 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:42.97739786 +0000 UTC m=+66.271464366 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs") pod "network-metrics-daemon-kjxws" (UID: "74a6aabd-b72d-422d-ac40-7d07a9e56a41") : secret "metrics-daemon-secret" not found
Oct 30 18:36:11.078505 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:11.077982    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w86dj\" (UniqueName: \"kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj\") pod \"network-check-target-k6kkr\" (UID: \"a7be31dd-dd7b-448e-938b-ba7a0807bf49\") " pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:11.081360 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:11.081333    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w86dj\" (UniqueName: \"kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj\") pod \"network-check-target-k6kkr\" (UID: \"a7be31dd-dd7b-448e-938b-ba7a0807bf49\") " pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:11.175031 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:11.174819    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:11.310795 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:11.310761    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:11.490154 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:11.490119    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cmf2j"]
Oct 30 18:36:11.494522 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:36:11.494492    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac85734e_3f97_473f_b3ab_8aadd898f381.slice/crio-6c97f9bda22eeb02b88ccb7e65301e762ba929e0195163087eccf0d92da587e5 WatchSource:0}: Error finding container 6c97f9bda22eeb02b88ccb7e65301e762ba929e0195163087eccf0d92da587e5: Status 404 returned error can't find the container with id 6c97f9bda22eeb02b88ccb7e65301e762ba929e0195163087eccf0d92da587e5
Oct 30 18:36:11.545070 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:11.544968    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cmf2j" event={"ID":"ac85734e-3f97-473f-b3ab-8aadd898f381","Type":"ContainerStarted","Data":"6c97f9bda22eeb02b88ccb7e65301e762ba929e0195163087eccf0d92da587e5"}
Oct 30 18:36:11.642784 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:11.642703    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k6kkr"]
Oct 30 18:36:11.646938 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:36:11.646907    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7be31dd_dd7b_448e_938b_ba7a0807bf49.slice/crio-fd3097b0bdebb315f864da8d33792c64b4a2f02544f222a4070b80d6093782d6 WatchSource:0}: Error finding container fd3097b0bdebb315f864da8d33792c64b4a2f02544f222a4070b80d6093782d6: Status 404 returned error can't find the container with id fd3097b0bdebb315f864da8d33792c64b4a2f02544f222a4070b80d6093782d6
Oct 30 18:36:12.549359 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:12.549314    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k6kkr" event={"ID":"a7be31dd-dd7b-448e-938b-ba7a0807bf49","Type":"ContainerStarted","Data":"fd3097b0bdebb315f864da8d33792c64b4a2f02544f222a4070b80d6093782d6"}
Oct 30 18:36:13.566452 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.566420    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n" event={"ID":"9e5ac899-ecba-4c42-aa7a-e34e14269efa","Type":"ContainerStarted","Data":"b9b64777cbe291c6661a8f706ef8d549047bdf72042bda4b47b4cdba443f056d"}
Oct 30 18:36:13.568233 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.568212    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/0.log"
Oct 30 18:36:13.568348 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.568252    2572 generic.go:356] "Generic (PLEG): container finished" podID="5cf855c4-4e7e-4669-9adc-583e22e27e73" containerID="775a49f779ca135178ef67f16febd403101948f8b7c56bec805747cf6cb805e4" exitCode=255
Oct 30 18:36:13.568348 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.568342    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm" event={"ID":"5cf855c4-4e7e-4669-9adc-583e22e27e73","Type":"ContainerDied","Data":"775a49f779ca135178ef67f16febd403101948f8b7c56bec805747cf6cb805e4"}
Oct 30 18:36:13.568565 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.568548    2572 scope.go:122] "RemoveContainer" containerID="775a49f779ca135178ef67f16febd403101948f8b7c56bec805747cf6cb805e4"
Oct 30 18:36:13.573316 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.573296    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg" event={"ID":"d2b52b9e-4797-4a2e-953e-7da0f61bca87","Type":"ContainerStarted","Data":"fef06f521cefc21fc9428e432194bfddcf361a1b2aa60bd9338d92cb92d590ee"}
Oct 30 18:36:13.575329 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.575309    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z2nsf" event={"ID":"272efc3d-7980-4fd1-a20d-9bd023f48bec","Type":"ContainerStarted","Data":"93e480114ea96b94e0ab649ca16814e07b56deb4aedab0b5a07887cbb1bcdf08"}
Oct 30 18:36:13.584456 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.584428    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-qhk9m" event={"ID":"0b90705d-6abb-481d-8af0-89e542709329","Type":"ContainerStarted","Data":"8f87af716388ad7b5d0e3079975706e1c5fd8938e44cee25b1b8ed342a6e83d1"}
Oct 30 18:36:13.587639 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.587612    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc" event={"ID":"4bee5094-7886-4fa9-aa7c-0ddc3d518619","Type":"ContainerStarted","Data":"9f316231f546c805d905e27b7a37bafbaf4997ac7e8ff8fc9f3be227e1e2f16e"}
Oct 30 18:36:13.636922 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.635772    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z2nsf" podStartSLOduration=1.80446691 podStartE2EDuration="7.635752554s" totalImagesPullingTime="5.831285644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:13.609707249 +0000 UTC m=+36.903773800" watchObservedRunningTime="2026-10-30 18:36:13.635752554 +0000 UTC m=+36.929819077"
Oct 30 18:36:13.685373 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.685314    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg" podStartSLOduration=203.496058701 podStartE2EDuration="3m28.685294949s" totalImagesPullingTime="5.189236248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:13.636956015 +0000 UTC m=+36.931022566" watchObservedRunningTime="2026-10-30 18:36:13.685294949 +0000 UTC m=+36.979361473"
Oct 30 18:36:13.686403 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.686364    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-qhk9m" podStartSLOduration=204.543107653 podStartE2EDuration="3m29.686350554s" totalImagesPullingTime="5.143242901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:13.6842427 +0000 UTC m=+36.978309248" watchObservedRunningTime="2026-10-30 18:36:13.686350554 +0000 UTC m=+36.980417076"
Oct 30 18:36:13.716405 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.716356    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc" podStartSLOduration=202.554053634 podStartE2EDuration="3m27.716327127s" totalImagesPullingTime="5.162273493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:13.710411301 +0000 UTC m=+37.004477821" watchObservedRunningTime="2026-10-30 18:36:13.716327127 +0000 UTC m=+37.010393651"
Oct 30 18:36:13.719339 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.719307    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8"]
Oct 30 18:36:13.744284 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.744262    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8"]
Oct 30 18:36:13.744609 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.744566    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8"
Oct 30 18:36:13.757521 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.757088    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:13.757521 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.757215    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 18:36:13.781561 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.781531    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zsxst\""
Oct 30 18:36:13.788242 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.787612    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:13.800061 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.795718    2572 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9czjj"]
Oct 30 18:36:13.820290 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.820269    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9czjj"]
Oct 30 18:36:13.820394 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.820384    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:13.833765 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.833730    2572 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:36:13.834219 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.834201    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:36:13.834541 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.834513    2572 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9fb5r\""
Oct 30 18:36:13.907802 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:13.907771    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79swd\" (UniqueName: \"kubernetes.io/projected/70da42f1-d38f-490e-8db0-4c1d141b2009-kube-api-access-79swd\") pod \"migrator-66df5d496b-4lrn8\" (UID: \"70da42f1-d38f-490e-8db0-4c1d141b2009\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8"
Oct 30 18:36:14.008853 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.008789    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79swd\" (UniqueName: \"kubernetes.io/projected/70da42f1-d38f-490e-8db0-4c1d141b2009-kube-api-access-79swd\") pod \"migrator-66df5d496b-4lrn8\" (UID: \"70da42f1-d38f-490e-8db0-4c1d141b2009\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8"
Oct 30 18:36:14.008853 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.008840    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-data-volume\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.009005 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.008923    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26kg9\" (UniqueName: \"kubernetes.io/projected/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-api-access-26kg9\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.009072 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.009055    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.009129 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.009088    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.009172 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.009140    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-crio-socket\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.010387 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.010312    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:14.020903 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.020876    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79swd\" (UniqueName: \"kubernetes.io/projected/70da42f1-d38f-490e-8db0-4c1d141b2009-kube-api-access-79swd\") pod \"migrator-66df5d496b-4lrn8\" (UID: \"70da42f1-d38f-490e-8db0-4c1d141b2009\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8"
Oct 30 18:36:14.109787 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.109752    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-data-volume\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.109945 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.109867    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26kg9\" (UniqueName: \"kubernetes.io/projected/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-api-access-26kg9\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.110026 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.110011    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.110085 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.110041    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.110085 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.110073    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-crio-socket\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.110217 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.110189    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-crio-socket\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.110506 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.110482    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-data-volume\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.110849 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.110829    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:36:14.110924 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.110895    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm podName:e7b7aaa9-68a9-4259-a9ce-001217dfa67f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:14.610875848 +0000 UTC m=+37.904942353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9czjj" (UID: "e7b7aaa9-68a9-4259-a9ce-001217dfa67f") : configmap "kube-rbac-proxy" not found
Oct 30 18:36:14.111127 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.111110    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:14.111236 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.111161    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls podName:e7b7aaa9-68a9-4259-a9ce-001217dfa67f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:14.611145953 +0000 UTC m=+37.905212458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9czjj" (UID: "e7b7aaa9-68a9-4259-a9ce-001217dfa67f") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:14.112706 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.112685    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8"
Oct 30 18:36:14.120060 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.120037    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26kg9\" (UniqueName: \"kubernetes.io/projected/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-api-access-26kg9\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.215258 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.215200    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:14.410123 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.410079    2572 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8"]
Oct 30 18:36:14.417778 ip-10-0-134-58 kubenswrapper[2572]: W1030 18:36:14.417747    2572 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70da42f1_d38f_490e_8db0_4c1d141b2009.slice/crio-82bd5c28c18ae081117ba6aeb96079c9aa67f6fea2f0dc9e057338aa6d805c33 WatchSource:0}: Error finding container 82bd5c28c18ae081117ba6aeb96079c9aa67f6fea2f0dc9e057338aa6d805c33: Status 404 returned error can't find the container with id 82bd5c28c18ae081117ba6aeb96079c9aa67f6fea2f0dc9e057338aa6d805c33
Oct 30 18:36:14.592625 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.592598    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/1.log"
Oct 30 18:36:14.593065 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.593047    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/0.log"
Oct 30 18:36:14.593145 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.593091    2572 generic.go:356] "Generic (PLEG): container finished" podID="5cf855c4-4e7e-4669-9adc-583e22e27e73" containerID="70c2560feead44db5071af9e4bce3fdbad88ffcafac5d9145f84a398128d94bb" exitCode=255
Oct 30 18:36:14.593245 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.593220    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm" event={"ID":"5cf855c4-4e7e-4669-9adc-583e22e27e73","Type":"ContainerDied","Data":"70c2560feead44db5071af9e4bce3fdbad88ffcafac5d9145f84a398128d94bb"}
Oct 30 18:36:14.593347 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.593286    2572 scope.go:122] "RemoveContainer" containerID="775a49f779ca135178ef67f16febd403101948f8b7c56bec805747cf6cb805e4"
Oct 30 18:36:14.593573 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.593558    2572 scope.go:122] "RemoveContainer" containerID="70c2560feead44db5071af9e4bce3fdbad88ffcafac5d9145f84a398128d94bb"
Oct 30 18:36:14.593758 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.593739    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-68flm_openshift-console-operator(5cf855c4-4e7e-4669-9adc-583e22e27e73)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm" podUID="5cf855c4-4e7e-4669-9adc-583e22e27e73"
Oct 30 18:36:14.595598 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.595213    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k6kkr" event={"ID":"a7be31dd-dd7b-448e-938b-ba7a0807bf49","Type":"ContainerStarted","Data":"d47c5e740882dbc72cd62926009b4b9bb6e52df642bce12224b97f98a34de039"}
Oct 30 18:36:14.595598 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.595344    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:14.596519 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.596490    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8" event={"ID":"70da42f1-d38f-490e-8db0-4c1d141b2009","Type":"ContainerStarted","Data":"82bd5c28c18ae081117ba6aeb96079c9aa67f6fea2f0dc9e057338aa6d805c33"}
Oct 30 18:36:14.615370 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.615347    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:14.615518 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.615501    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.615573 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.615515    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:14.615573 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.615533    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6684c84797-p6964: secret "image-registry-tls" not found
Oct 30 18:36:14.615573 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.615536    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:14.615700 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.615585    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls podName:e32eb376-a16f-421c-8958-a410a5650964 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:22.615567738 +0000 UTC m=+45.909634244 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls") pod "image-registry-6684c84797-p6964" (UID: "e32eb376-a16f-421c-8958-a410a5650964") : secret "image-registry-tls" not found
Oct 30 18:36:14.615700 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.615601    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:36:14.615700 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.615640    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:14.615700 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.615656    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm podName:e7b7aaa9-68a9-4259-a9ce-001217dfa67f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:15.615639972 +0000 UTC m=+38.909706486 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9czjj" (UID: "e7b7aaa9-68a9-4259-a9ce-001217dfa67f") : configmap "kube-rbac-proxy" not found
Oct 30 18:36:14.615700 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.615683    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls podName:e7b7aaa9-68a9-4259-a9ce-001217dfa67f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:15.615671183 +0000 UTC m=+38.909737698 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9czjj" (UID: "e7b7aaa9-68a9-4259-a9ce-001217dfa67f") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:14.631316 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.631273    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n" podStartSLOduration=198.842120269 podStartE2EDuration="3m24.631261402s" totalImagesPullingTime="5.789141133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:14.630733018 +0000 UTC m=+37.924799540" watchObservedRunningTime="2026-10-30 18:36:14.631261402 +0000 UTC m=+37.925327924"
Oct 30 18:36:14.647498 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.647451    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k6kkr" podStartSLOduration=35.866939556 podStartE2EDuration="37.647437218s" totalImagesPullingTime="1.780497662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:35:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:14.646271289 +0000 UTC m=+37.940337811" watchObservedRunningTime="2026-10-30 18:36:14.647437218 +0000 UTC m=+37.941503740"
Oct 30 18:36:14.716745 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.716687    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:14.716745 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.716719    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:14.716745 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.716739    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:14.716982 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.716797    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:14.716982 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.716851    2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:36:14.716982 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.716854    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:14.716982 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.716946    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:36:14.717119 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.716979    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:14.717119 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.717066    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:14.717228 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.717153    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls podName:734db718-c901-4fe7-a8e1-b90823ccd66c nodeName:}" failed. No retries permitted until 2026-10-30 18:36:22.717137717 +0000 UTC m=+46.011204222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls") pod "dns-default-jcn7p" (UID: "734db718-c901-4fe7-a8e1-b90823ccd66c") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:14.717228 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.717217    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:14.717421 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.717406    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:22.7173923 +0000 UTC m=+46.011458822 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : configmap references non-existent config key: service-ca.crt
Oct 30 18:36:14.717661 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.717643    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:14.717696 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.717668    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f45597c6-hfv8h: secret "image-registry-tls" not found
Oct 30 18:36:14.717728 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.717705    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls podName:cd26f706-5971-4c47-a63e-699e7962ef8e nodeName:}" failed. No retries permitted until 2026-10-30 18:36:22.717692511 +0000 UTC m=+46.011759012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls") pod "image-registry-76f45597c6-hfv8h" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e") : secret "image-registry-tls" not found
Oct 30 18:36:14.717777 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.717763    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:14.717809 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.717803    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls podName:f0ba138d-0a24-4f77-b1f4-c0861a48797f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:22.717792766 +0000 UTC m=+46.011859301 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n92pf" (UID: "f0ba138d-0a24-4f77-b1f4-c0861a48797f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:14.717843 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.717820    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls podName:9e9032e7-1ec1-4a5d-8e28-622d4fd01392 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:22.717812675 +0000 UTC m=+46.011879175 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-vzrmj" (UID: "9e9032e7-1ec1-4a5d-8e28-622d4fd01392") : secret "samples-operator-tls" not found
Oct 30 18:36:14.718169 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.718145    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:22.718129082 +0000 UTC m=+46.012195582 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : secret "router-metrics-certs-default" not found
Oct 30 18:36:14.718284 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.718212    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:36:14.718284 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:14.718263    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert podName:f478aac2-de91-4b13-836b-ba09809222f0 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:22.718247881 +0000 UTC m=+46.012314381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k9dgt" (UID: "f478aac2-de91-4b13-836b-ba09809222f0") : secret "networking-console-plugin-cert" not found
Oct 30 18:36:14.728603 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.728569    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:14.913672 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:14.913635    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:15.482461 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:15.482429    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:15.602837 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:15.602810    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/1.log"
Oct 30 18:36:15.603310 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:15.603214    2572 scope.go:122] "RemoveContainer" containerID="70c2560feead44db5071af9e4bce3fdbad88ffcafac5d9145f84a398128d94bb"
Oct 30 18:36:15.603468 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:15.603442    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-68flm_openshift-console-operator(5cf855c4-4e7e-4669-9adc-583e22e27e73)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm" podUID="5cf855c4-4e7e-4669-9adc-583e22e27e73"
Oct 30 18:36:15.625545 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:15.625510    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:15.625707 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:15.625550    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:15.625866 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:15.625840    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:15.625984 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:15.625908    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls podName:e7b7aaa9-68a9-4259-a9ce-001217dfa67f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:17.625892409 +0000 UTC m=+40.919958909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9czjj" (UID: "e7b7aaa9-68a9-4259-a9ce-001217dfa67f") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:15.626181 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:15.626167    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:36:15.626250 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:15.626216    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm podName:e7b7aaa9-68a9-4259-a9ce-001217dfa67f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:17.62620292 +0000 UTC m=+40.920269420 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9czjj" (UID: "e7b7aaa9-68a9-4259-a9ce-001217dfa67f") : configmap "kube-rbac-proxy" not found
Oct 30 18:36:16.608669 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:16.608633    2572 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8" event={"ID":"70da42f1-d38f-490e-8db0-4c1d141b2009","Type":"ContainerStarted","Data":"d593f0bb40719631cdcac32826175ccfd140334ba03568b91a700444d35534ca"}
Oct 30 18:36:16.625744 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:16.625702    2572 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8" podStartSLOduration=2.055529508 podStartE2EDuration="3.625688745s" totalImagesPullingTime="1.570159237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:16.625301766 +0000 UTC m=+39.919368314" watchObservedRunningTime="2026-10-30 18:36:16.625688745 +0000 UTC m=+39.919755266"
Oct 30 18:36:17.232544 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:17.232517    2572 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:17.232716 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:17.232580    2572 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:17.232913 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:17.232899    2572 scope.go:122] "RemoveContainer" containerID="70c2560feead44db5071af9e4bce3fdbad88ffcafac5d9145f84a398128d94bb"
Oct 30 18:36:17.233114 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:17.233079    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-68flm_openshift-console-operator(5cf855c4-4e7e-4669-9adc-583e22e27e73)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm" podUID="5cf855c4-4e7e-4669-9adc-583e22e27e73"
Oct 30 18:36:17.644191 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:17.644118    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:17.644191 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:17.644152    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:17.644568 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:17.644214    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:36:17.644568 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:17.644245    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:17.644568 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:17.644293    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm podName:e7b7aaa9-68a9-4259-a9ce-001217dfa67f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:21.644272597 +0000 UTC m=+44.938339100 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9czjj" (UID: "e7b7aaa9-68a9-4259-a9ce-001217dfa67f") : configmap "kube-rbac-proxy" not found
Oct 30 18:36:17.644568 ip-10-0-134-58 kubenswrapper[2572]: E1030 18:36:17.644316    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls podName:e7b7aaa9-68a9-4259-a9ce-001217dfa67f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:21.644304732 +0000 UTC m=+44.938371235 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9czjj" (UID: "e7b7aaa9-68a9-4259-a9ce-001217dfa67f") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:18.227003 ip-10-0-134-58 kubenswrapper[2572]: I1030 18:36:18.226835    2572 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:36:18.227000 ip-10-0-134-58 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 18:36:18.235479 ip-10-0-134-58 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 18:36:18.235694 ip-10-0-134-58 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 18:36:18.235754 ip-10-0-134-58 systemd[1]: kubelet.service: Consumed 2.944s CPU time.
Oct 30 18:36:18.244775 ip-10-0-134-58 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:36:18.336383 ip-10-0-134-58 kubenswrapper[4925]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:36:18.336383 ip-10-0-134-58 kubenswrapper[4925]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:36:18.336383 ip-10-0-134-58 kubenswrapper[4925]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:36:18.336383 ip-10-0-134-58 kubenswrapper[4925]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:36:18.338377 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338304    4925 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:18.338377 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338377    4925 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338381    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338384    4925 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338388    4925 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338391    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338393    4925 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338396    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338399    4925 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338401    4925 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338404    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338407    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338409    4925 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338412    4925 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338414    4925 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338417    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338421    4925 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338423    4925 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338426    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338428    4925 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338434    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:18.338442 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338438    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338440    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338443    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338446    4925 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338449    4925 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338452    4925 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338454    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338457    4925 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338460    4925 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338462    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338465    4925 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338468    4925 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338470    4925 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338473    4925 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338475    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338478    4925 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338481    4925 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338483    4925 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338492    4925 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:18.338947 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338495    4925 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338499    4925 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338503    4925 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338507    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338511    4925 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338514    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338516    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338519    4925 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338522    4925 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338525    4925 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338527    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338529    4925 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338532    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338534    4925 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338538    4925 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338541    4925 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338543    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338546    4925 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338549    4925 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338552    4925 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:18.339445 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338554    4925 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338557    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338561    4925 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338565    4925 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338568    4925 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338571    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338573    4925 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338576    4925 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338579    4925 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338581    4925 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338588    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338591    4925 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338594    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338597    4925 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338599    4925 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338602    4925 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338604    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338607    4925 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338609    4925 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:18.339936 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338612    4925 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338614    4925 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338617    4925 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338619    4925 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338622    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338624    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338627    4925 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338629    4925 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338632    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338635    4925 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338637    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338640    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338642    4925 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338645    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338647    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338649    4925 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338652    4925 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338654    4925 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338657    4925 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:18.340428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338659    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338662    4925 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338664    4925 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338667    4925 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338671    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338673    4925 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338676    4925 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.338679    4925 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339430    4925 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339438    4925 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339440    4925 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339443    4925 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339446    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339449    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339458    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339462    4925 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339464    4925 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339467    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339470    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339472    4925 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:18.340912 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339475    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339477    4925 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339480    4925 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339482    4925 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339485    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339487    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339490    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339495    4925 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339500    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339503    4925 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339506    4925 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339509    4925 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339511    4925 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339514    4925 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339516    4925 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339518    4925 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339522    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339524    4925 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339527    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339529    4925 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:18.341450 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339532    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339535    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339537    4925 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339540    4925 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339542    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339544    4925 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339547    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339549    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339552    4925 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339555    4925 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339557    4925 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339560    4925 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339562    4925 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339565    4925 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339568    4925 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339570    4925 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339573    4925 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339578    4925 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339582    4925 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:18.341959 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339585    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339588    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339591    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339593    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339596    4925 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339598    4925 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339601    4925 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339603    4925 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339606    4925 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339609    4925 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339611    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339613    4925 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339616    4925 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339618    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339621    4925 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339624    4925 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339626    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339629    4925 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339631    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:18.342451 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339634    4925 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339636    4925 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339639    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339642    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339645    4925 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339647    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339650    4925 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339653    4925 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339656    4925 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339658    4925 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339661    4925 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339663    4925 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339666    4925 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339668    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339671    4925 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339674    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339676    4925 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339678    4925 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339681    4925 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339683    4925 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:18.342928 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339686    4925 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339688    4925 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339691    4925 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339693    4925 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339695    4925 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339698    4925 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339700    4925 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339703    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339706    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339708    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339711    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339714    4925 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339716    4925 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339719    4925 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339722    4925 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.339725    4925 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340187    4925 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340193    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340196    4925 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:18.343428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340199    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340201    4925 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340204    4925 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340207    4925 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340209    4925 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340212    4925 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340215    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340217    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340220    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340223    4925 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340225    4925 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340228    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340230    4925 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340233    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340235    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340238    4925 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340243    4925 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340246    4925 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340249    4925 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:18.343935 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340251    4925 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340256    4925 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340259    4925 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340263    4925 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340266    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340269    4925 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340271    4925 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340273    4925 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340276    4925 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340279    4925 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340281    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340284    4925 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340286    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340289    4925 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340291    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340294    4925 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340296    4925 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340299    4925 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340301    4925 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:18.344430 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340304    4925 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340306    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340309    4925 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340311    4925 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340314    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340316    4925 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340319    4925 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340323    4925 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340325    4925 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340328    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340332    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340335    4925 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340337    4925 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340340    4925 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340342    4925 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340345    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340348    4925 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340351    4925 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340353    4925 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340355    4925 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:18.344925 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340358    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340360    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340363    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340366    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340368    4925 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340371    4925 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340373    4925 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340376    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340379    4925 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340381    4925 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340384    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340386    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340389    4925 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340391    4925 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340394    4925 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340396    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340399    4925 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340401    4925 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340404    4925 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340406    4925 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:18.345444 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340409    4925 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340411    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340415    4925 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340418    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340421    4925 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340423    4925 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340426    4925 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340428    4925 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340431    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340434    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340437    4925 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340439    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340442    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340445    4925 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340447    4925 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340451    4925 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340455    4925 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340458    4925 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340461    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:18.345954 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340463    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340466    4925 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340468    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340471    4925 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340474    4925 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.340476    4925 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340567    4925 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340574    4925 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340579    4925 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340584    4925 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340598    4925 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340601    4925 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340606    4925 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340610    4925 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340613    4925 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340618    4925 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340622    4925 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340625    4925 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340628    4925 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340631    4925 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340633    4925 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340636    4925 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340639    4925 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:36:18.346431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340643    4925 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340646    4925 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340649    4925 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340652    4925 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340655    4925 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340658    4925 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340662    4925 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340665    4925 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340668    4925 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340671    4925 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340674    4925 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340677    4925 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340680    4925 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340684    4925 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340686    4925 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340690    4925 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340693    4925 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340696    4925 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340699    4925 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340702    4925 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340705    4925 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340709    4925 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340713    4925 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340716    4925 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:36:18.347134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340720    4925 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340723    4925 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340726    4925 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340729    4925 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340733    4925 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340736    4925 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340738    4925 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340741    4925 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340744    4925 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340747    4925 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340750    4925 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340753    4925 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340755    4925 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340759    4925 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340763    4925 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340766    4925 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340769    4925 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340772    4925 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340775    4925 flags.go:64] FLAG: --help="false"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340778    4925 flags.go:64] FLAG: --hostname-override="ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340781    4925 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340784    4925 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340787    4925 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:36:18.348037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340790    4925 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340794    4925 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340796    4925 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340799    4925 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340802    4925 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340804    4925 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340807    4925 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340810    4925 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340813    4925 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340817    4925 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340820    4925 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340823    4925 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340826    4925 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340829    4925 flags.go:64] FLAG: --lock-file=""
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340832    4925 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340835    4925 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340838    4925 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340843    4925 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340846    4925 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340850    4925 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340852    4925 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340855    4925 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340859    4925 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340862    4925 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340865    4925 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:36:18.348712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340869    4925 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340872    4925 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340876    4925 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340879    4925 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340882    4925 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340885    4925 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340888    4925 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340891    4925 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340894    4925 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340897    4925 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340903    4925 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340906    4925 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340909    4925 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340912    4925 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340915    4925 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340918    4925 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340923    4925 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340926    4925 flags.go:64] FLAG: --port="10250"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340929    4925 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340932    4925 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-09a53192a76a17ddb"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340935    4925 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340938    4925 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340941    4925 flags.go:64] FLAG: --register-node="true"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340944    4925 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340948    4925 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340951    4925 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:36:18.349352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340954    4925 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340957    4925 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340961    4925 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340963    4925 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340966    4925 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340969    4925 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340973    4925 flags.go:64] FLAG: --runonce="false"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340976    4925 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340979    4925 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340982    4925 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340985    4925 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340988    4925 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340991    4925 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340994    4925 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.340997    4925 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341000    4925 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341003    4925 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341006    4925 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341009    4925 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341012    4925 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341014    4925 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341017    4925 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341024    4925 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341027    4925 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341030    4925 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:36:18.349989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341034    4925 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341036    4925 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341039    4925 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341042    4925 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341044    4925 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341047    4925 flags.go:64] FLAG: --v="2"
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341052    4925 flags.go:64] FLAG: --version="false"
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341056    4925 flags.go:64] FLAG: --vmodule=""
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341060    4925 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341063    4925 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341237    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341242    4925 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341245    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341251    4925 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341254    4925 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341256    4925 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341259    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341262    4925 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341264    4925 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341267    4925 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341269    4925 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341272    4925 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:18.350642 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341274    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341277    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341280    4925 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341282    4925 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341285    4925 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341288    4925 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341290    4925 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341294    4925 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341297    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341299    4925 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341302    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341305    4925 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341307    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341311    4925 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341314    4925 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341317    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341320    4925 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341323    4925 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341325    4925 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:18.351758 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341328    4925 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341331    4925 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341334    4925 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341336    4925 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341340    4925 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341343    4925 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341346    4925 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341350    4925 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341354    4925 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341356    4925 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341359    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341362    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341365    4925 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341367    4925 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341370    4925 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341372    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341375    4925 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341378    4925 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341380    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:18.352447 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341383    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341386    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341389    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341391    4925 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341394    4925 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341396    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341399    4925 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341402    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341404    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341407    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341409    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341411    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341414    4925 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341416    4925 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341419    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341422    4925 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341425    4925 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341429    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341431    4925 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341434    4925 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:18.352984 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341436    4925 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341439    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341441    4925 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341444    4925 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341446    4925 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341449    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341451    4925 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341454    4925 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341457    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341459    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341462    4925 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341464    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341467    4925 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341470    4925 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341473    4925 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341475    4925 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341478    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341480    4925 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341482    4925 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:18.353502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341485    4925 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341487    4925 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341490    4925 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341492    4925 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341495    4925 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341497    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341500    4925 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341502    4925 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341505    4925 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341508    4925 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341512    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341514    4925 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341516    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341519    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341521    4925 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341524    4925 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:18.354122 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341526    4925 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.341552    4925 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341591    4925 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341595    4925 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341598    4925 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341601    4925 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341604    4925 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341606    4925 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341609    4925 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341611    4925 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341614    4925 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341618    4925 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341622    4925 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341624    4925 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341627    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341629    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:18.354621 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341632    4925 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341634    4925 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341637    4925 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341639    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341642    4925 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341644    4925 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341646    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341649    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341651    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341654    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341658    4925 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341661    4925 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341663    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341666    4925 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341668    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341671    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341673    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341676    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341678    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341680    4925 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:18.355074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341683    4925 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341685    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341688    4925 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341690    4925 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341692    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341695    4925 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341697    4925 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341700    4925 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341703    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341705    4925 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341708    4925 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341710    4925 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341713    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341715    4925 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341717    4925 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341720    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341722    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341725    4925 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341727    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:18.355638 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341729    4925 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341732    4925 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341734    4925 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341738    4925 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341741    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341744    4925 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341746    4925 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341749    4925 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341751    4925 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341754    4925 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341756    4925 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341758    4925 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341761    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341763    4925 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341766    4925 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341769    4925 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341772    4925 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341774    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341776    4925 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341779    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:18.356126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341781    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341784    4925 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341786    4925 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341789    4925 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341791    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341794    4925 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341796    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341799    4925 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341801    4925 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341804    4925 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341806    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341808    4925 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341811    4925 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341813    4925 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341816    4925 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341820    4925 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341823    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341826    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341829    4925 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341832    4925 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:18.356650 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341834    4925 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341837    4925 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341839    4925 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341842    4925 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341844    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341847    4925 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341849    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341852    4925 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341854    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341859    4925 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341862    4925 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341865    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.341868    4925 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.345443    4925 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.345456    4925 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.345474    4925 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.345478    4925 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345529    4925 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345535    4925 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:18.357172 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345538    4925 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345541    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345544    4925 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345546    4925 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345549    4925 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345552    4925 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345554    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345557    4925 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345559    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345562    4925 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345565    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345568    4925 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345570    4925 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345573    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345576    4925 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345579    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345581    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345584    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345586    4925 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:18.357635 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345590    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345592    4925 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345595    4925 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345598    4925 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345600    4925 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345603    4925 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345605    4925 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345607    4925 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345610    4925 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345612    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345615    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345617    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345620    4925 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345622    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345624    4925 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345627    4925 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345629    4925 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345632    4925 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345634    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345637    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:18.358078 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345640    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345642    4925 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345645    4925 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345647    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345651    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345653    4925 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345656    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345658    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345661    4925 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345663    4925 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345666    4925 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345668    4925 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345671    4925 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345674    4925 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345676    4925 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345679    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345682    4925 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345684    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345687    4925 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345689    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:18.358580 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345692    4925 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345694    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345696    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345699    4925 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345701    4925 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345704    4925 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345706    4925 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345708    4925 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345711    4925 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345713    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345715    4925 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345718    4925 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345721    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345724    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345726    4925 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345729    4925 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345731    4925 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345734    4925 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345737    4925 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345739    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:18.359064 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345742    4925 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345744    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345747    4925 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345749    4925 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345752    4925 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345754    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345757    4925 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345759    4925 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345762    4925 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345764    4925 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345766    4925 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345769    4925 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345771    4925 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345775    4925 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345780    4925 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345783    4925 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345786    4925 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345789    4925 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345792    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:18.359570 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345795    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345798    4925 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345801    4925 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345804    4925 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345807    4925 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345809    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.345837    4925 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345967    4925 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345971    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345974    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345976    4925 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345979    4925 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345982    4925 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345985    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345987    4925 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345990    4925 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:18.360030 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345993    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345995    4925 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.345999    4925 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346003    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346006    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346009    4925 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346011    4925 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346014    4925 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346016    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346019    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346021    4925 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346023    4925 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346026    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346028    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346031    4925 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346033    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346035    4925 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346038    4925 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346040    4925 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:18.360448 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346043    4925 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346045    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346047    4925 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346050    4925 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346052    4925 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346055    4925 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346057    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346060    4925 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346062    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346065    4925 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346068    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346070    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346073    4925 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346075    4925 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346078    4925 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346080    4925 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346083    4925 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346085    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346087    4925 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:18.361015 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346090    4925 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346110    4925 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346114    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346118    4925 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346120    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346123    4925 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346125    4925 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346128    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346131    4925 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346133    4925 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346136    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346138    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346141    4925 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346143    4925 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346145    4925 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346148    4925 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346150    4925 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346153    4925 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346157    4925 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346160    4925 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:18.361563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346162    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346165    4925 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346168    4925 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346171    4925 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346174    4925 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346177    4925 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346180    4925 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346182    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346184    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346187    4925 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346190    4925 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346192    4925 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346194    4925 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346197    4925 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346199    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346202    4925 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346204    4925 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346207    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346209    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:18.362044 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346212    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346214    4925 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346217    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346219    4925 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346222    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346224    4925 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346226    4925 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346229    4925 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346233    4925 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346237    4925 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346240    4925 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346244    4925 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346247    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346249    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346252    4925 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346255    4925 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346257    4925 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346260    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346263    4925 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:18.362705 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346265    4925 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.346289    4925 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346319    4925 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346323    4925 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346326    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346329    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346331    4925 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346334    4925 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346336    4925 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346339    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346341    4925 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346343    4925 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346346    4925 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346349    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346351    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346354    4925 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:18.363456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346356    4925 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346359    4925 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346361    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346364    4925 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346367    4925 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346369    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346372    4925 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346375    4925 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346378    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346383    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346386    4925 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346389    4925 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346393    4925 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346395    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346398    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346404    4925 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346407    4925 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346410    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346412    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:18.364065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346415    4925 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346417    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346420    4925 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346422    4925 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346425    4925 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346427    4925 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346429    4925 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346432    4925 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346435    4925 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346437    4925 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346440    4925 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346443    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346445    4925 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346448    4925 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346451    4925 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346453    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346456    4925 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346458    4925 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346461    4925 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:18.364896 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346463    4925 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346466    4925 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346468    4925 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346472    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346474    4925 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346477    4925 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346479    4925 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346482    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346484    4925 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346487    4925 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346490    4925 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346493    4925 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346496    4925 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346499    4925 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346501    4925 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346504    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346506    4925 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346509    4925 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346511    4925 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346514    4925 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:18.365611 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346516    4925 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346519    4925 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346521    4925 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346524    4925 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346526    4925 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346528    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346531    4925 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346534    4925 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346536    4925 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346539    4925 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346541    4925 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346544    4925 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346546    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346549    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346551    4925 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346556    4925 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346559    4925 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346561    4925 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346564    4925 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346567    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:18.366293 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346569    4925 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346572    4925 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346576    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346579    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346582    4925 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346584    4925 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346587    4925 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346590    4925 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346592    4925 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346594    4925 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346597    4925 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346599    4925 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346604    4925 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:18.346607    4925 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.346652    4925 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.348709    4925 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.348766    4925 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.348957    4925 server.go:1014] "Starting client certificate rotation"
Oct 30 18:36:18.367171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.349070    4925 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:36:18.367955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.349127    4925 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:30:37 +0000 UTC" deadline="2028-05-01 15:37:34.698340274 +0000 UTC"
Oct 30 18:36:18.367955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.349162    4925 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13173h1m16.349180097s"
Oct 30 18:36:18.367955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.349399    4925 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:36:18.367955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.350530    4925 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:36:18.367955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.351955    4925 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:36:18.367955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.353651    4925 log.go:25] "Validated CRI v1 image API"
Oct 30 18:36:18.367955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.354314    4925 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:36:18.367955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.356063    4925 fs.go:135] Filesystem UUIDs: map[1fe97406-82cc-4c68-9e3b-678097457625:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b090eb49-0213-406a-8cfd-4a907394034e:/dev/nvme0n1p3]
Oct 30 18:36:18.368366 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.356083    4925 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/05882ddb5a0f5813f31f6de21cef09f1ccbd8629b246f700e925c91beb5340f0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/05882ddb5a0f5813f31f6de21cef09f1ccbd8629b246f700e925c91beb5340f0/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/518b6b63e2b339f44b2facd01785314e1b0649197ae8ea149933195c63c6821c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/518b6b63e2b339f44b2facd01785314e1b0649197ae8ea149933195c63c6821c/userdata/shm major:0 minor:237 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5886c3667fcb917c58b17d5a153971827aea2790497f54c1a53f6d71ad31dd84/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5886c3667fcb917c58b17d5a153971827aea2790497f54c1a53f6d71ad31dd84/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5a42aedc4ebe3ce8adbed1030f1b5994ef0e8cc402352acf984b72350f25a3b1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5a42aedc4ebe3ce8adbed1030f1b5994ef0e8cc402352acf984b72350f25a3b1/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6ab635073ce18d9860646ba15f6fd95ea3ec30b03f52d9724794589a435d01b9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6ab635073ce18d9860646ba15f6fd95ea3ec30b03f52d9724794589a435d01b9/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6c97f9bda22eeb02b88ccb7e65301e762ba929e0195163087eccf0d92da587e5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6c97f9bda22eeb02b88ccb7e65301e762ba929e0195163087eccf0d92da587e5/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7e71a6838390fe0ef8b0685088c12fec304c859abebc15e65eb0ef7a0f650886/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7e71a6838390fe0ef8b0685088c12fec304c859abebc15e65eb0ef7a0f650886/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/82bd5c28c18ae081117ba6aeb96079c9aa67f6fea2f0dc9e057338aa6d805c33/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/82bd5c28c18ae081117ba6aeb96079c9aa67f6fea2f0dc9e057338aa6d805c33/userdata/shm major:0 minor:328 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8aed94ba983787ab53fa85bf2c419096c9b05b013a9bbd1746f293ec8216d2d0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8aed94ba983787ab53fa85bf2c419096c9b05b013a9bbd1746f293ec8216d2d0/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8ebf01bb27e75d8f710025340685c56c55877a45e266253db115f25358a5b0de/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8ebf01bb27e75d8f710025340685c56c55877a45e266253db115f25358a5b0de/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9044b821157b9dd8b87564d99f0be80e50cabac7d4c89cb19da361f37738104c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9044b821157b9dd8b87564d99f0be80e50cabac7d4c89cb19da361f37738104c/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/abd8ee6926056d9e202f1f341f371a7d577561fc4fc8005f5652ff22ac4fb453/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/abd8ee6926056d9e202f1f341f371a7d577561fc4fc8005f5652ff22ac4fb453/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c21c20d181f19eb90e9a3db0649f647bd4f23ceeaf4fd9e4a9bdde086bb18043/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c21c20d181f19eb90e9a3db0649f647bd4f23ceeaf4fd9e4a9bdde086bb18043/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c3fe00b3d12767cf8289d90c5eaeae0d214b23f5e82fb05d7a22cbd0da0cf4de/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c3fe00b3d12767cf8289d90c5eaeae0d214b23f5e82fb05d7a22cbd0da0cf4de/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c45e8b0359365cc553682db8d420ee0da63380c7193b24dcf96d959336d76970/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c45e8b0359365cc553682db8d420ee0da63380c7193b24dcf96d959336d76970/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c4f431637bd8ea72fd98b092b53df10b00f0d0a3ba3a856b957e0501a506bece/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c4f431637bd8ea72fd98b092b53df10b00f0d0a3ba3a856b957e0501a506bece/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d34814233153f704c3a04db0cd80b7994903b3e1dc582de690be12869ca63296/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d34814233153f704c3a04db0cd80b7994903b3e1dc582de690be12869ca63296/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e7aab5272fcd7c97b395509f63b18d1012b5eec1aeae7c72965a6bce0c3089ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e7aab5272fcd7c97b395509f63b18d1012b5eec1aeae7c72965a6bce0c3089ae/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eb46efb03bca9a23e02ad293848408bc0d72301fe54b8942d15b471d323bc9b5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eb46efb03bca9a23e02ad293848408bc0d72301fe54b8942d15b471d323bc9b5/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f35a1bc16f4abc1d8257b702bc5da0459be08a98e1f4ff005bd0693db0140d57/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f35a1bc16f4abc1d8257b702bc5da0459be08a98e1f4ff005bd0693db0140d57/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/faa1dd80e2e92bf9d05e3aea03b5bd5733995ad7888281b0af8b4079958531c0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/faa1dd80e2e92bf9d05e3aea03b5bd5733995ad7888281b0af8b4079958531c0/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fd3097b0bdebb315f864da8d33792c64b4a2f02544f222a4070b80d6093782d6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fd3097b0bdebb315f864da8d33792c64b4a2f02544f222a4070b80d6093782d6/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fda3295ed6547c21104bce3ea46135cc2b26d2e328d9d58fca1a66beea3b5b97/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fda3295ed6547c21104bce3ea46135cc2b26d2e328d9d58fca1a66beea3b5b97/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b90705d-6abb-481d-8af0-89e542709329/volumes/kubernetes.io~projected/kube-api-access-7dn5r:{mountpoint:/var/lib/kubelet/pods/0b90705d-6abb-481d-8af0-89e542709329/volumes/kubernetes.io~projected/kube-api-access-7dn5r major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b90705d-6abb-481d-8af0-89e542709329/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/0b90705d-6abb-481d-8af0-89e542709329/volumes/kubernetes.io~secret/serving-cert major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/272efc3d-7980-4fd1-a20d-9bd023f48bec/volumes/kubernetes.io~projected/kube-api-access-dkr9h:{mountpoint:/var/lib/kubelet/pods/272efc3d-7980-4fd1-a20d-9bd023f48bec/volumes/kubernetes.io~projected/kube-api-access-dkr9h major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/272efc3d-7980-4fd1-a20d-9bd023f48bec/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/272efc3d-7980-4fd1-a20d-9bd023f48bec/volumes/kubernetes.io~secret/cert major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c77e172-a80d-4eda-986d-ec8c3c52e131/volumes/kubernetes.io~projected/kube-api-access-f9mtz:{mountpoint:/var/lib/kubelet/pods/2c77e172-a80d-4eda-986d-ec8c3c52e131/volumes/kubernetes.io~projected/kube-api-access-f9mtz major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c77e172-a80d-4eda-986d-ec8c3c52e131/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/2c77e172-a80d-4eda-986d-ec8c3c52e131/volumes/kubernetes.io~secret/default-certificate major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2c77e172-a80d-4eda-986d-ec8c3c52e131/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/2c77e172-a80d-4eda-986d-ec8c3c52e131/volumes/kubernetes.io~secret/stats-auth major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/36d759c0-ed6e-4170-aa8f-70560074a467/volumes/kubernetes.io~projected/kube-api-access-6fn8p:{mountpoint:/var/lib/kubelet/pods/36d759c0-ed6e-4170-aa8f-70560074a467/volumes/kubernetes.io~projected/kube-api-access-6fn8p major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3762f795-944b-458e-9458-c26eaf133871/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/3762f795-944b-458e-9458-c26eaf133871/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3e7518c9-9163-41ac-9546-556b1773cec6/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/3e7518c9-9163-41ac-9546-556b1773cec6/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3e7518c9-9163-41ac-9546-556b1773cec6/volumes/kubernetes.io~projected/kube-api-access-vgqmm:{mountpoint:/var/lib/kubelet/pods/3e7518c9-9163-41ac-9546-556b1773cec6/volumes/kubernetes.io~projected/kube-api-access-vgqmm major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3e7518c9-9163-41ac-9546-556b1773cec6/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/3e7518c9-9163-41ac-9546-556b1773cec6/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4bee5094-7886-4fa9-aa7c-0ddc3d518619/volumes/kubernetes.io~projected/kube-api-access-84gss:{mountpoint:/var/lib/kubelet/pods/4bee5094-7886-4fa9-aa7c-0ddc3d518619/volumes/kubernetes.io~projected/kube-api-access-84gss major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4bee5094-7886-4fa9-aa7c-0ddc3d518619/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/4bee5094-7886-4fa9-aa7c-0ddc3d518619/volumes/kubernetes.io~secret/serving-cert major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4c1460ce-789a-411c-b748-2b3506811aa1/volumes/kubernetes.io~projected/kube-api-access-kpvfc:{mountpoint:/var/lib/kubelet/pods/4c1460ce-789a-411c-b748-2b3506811aa1/volumes/kubernetes.io~projected/kube-api-access-kpvfc major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4cd65688-51e1-4b8c-8dc5-6e3771a74e66/volumes/kubernetes.io~projected/kube-api-access-jms4w:{mountpoint:/var/lib/kubelet/pods/4cd65688-51e1-4b8c-8dc5-6e3771a74e66/volumes/kubernetes.io~projected/kube-api-access-jms4w major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5cf855c4-4e7e-4669-9adc-583e22e27e73/volumes/kubernetes.io~projected/kube-api-access-tl4tg:{mountpoint:/var/lib/kubelet/pods/5cf855c4-4e7e-4669-9adc-583e22e27e73/volumes/kubernetes.io~projected/kube-api-access-tl4tg major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5cf855c4-4e7e-4669-9adc-583e22e27e73/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/5cf855c4-4e7e-4669-9adc-583e22e27e73/volumes/kubernetes.io~secret/serving-cert major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/628d4dcc-30b9-4817-815e-53d36818b7c2/volumes/kubernetes.io~projected/kube-api-access-k7bxj:{mountpoint:/var/lib/kubelet/pods/628d4dcc-30b9-4817-815e-53d36818b7c2/volumes/kubernetes.io~projected/kube-api-access-k7bxj major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70da42f1-d38f-490e-8db0-4c1d141b2009/volumes/kubernetes.io~projected/kube-api-access-79swd:{mountpoint:/var/lib/kubelet/pods/70da42f1-d38f-490e-8db0-4c1d141b2009/volumes/kubernetes.io~projected/kube-api-access-79swd major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/734db718-c901-4fe7-a8e1-b90823ccd66c/volumes/kubernetes.io~projected/kube-api-access-s9q7l:{mountpoint:/var/lib/kubelet/pods/734db718-c901-4fe7-a8e1-b90823ccd66c/volumes/kubernetes.io~projected/kube-api-access-s9q7l major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/74a6aabd-b72d-422d-ac40-7d07a9e56a41/volumes/kubernetes.io~projected/kube-api-access-z9ndr:{mountpoint:/var/lib/kubelet/pods/74a6aabd-b72d-422d-ac40-7d07a9e56a41/volumes/kubernetes.io~projected/kube-api-access-z9ndr major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7563c219-088f-4c66-81f5-86c581f08f34/volumes/kubernetes.io~projected/kube-api-access-ztnjp:{mountpoint:/var/lib/kubelet/pods/7563c219-088f-4c66-81f5-86c581f08f34/volumes/kubernetes.io~projected/kube-api-access-ztnjp major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/800f854c-c40c-45d9-9ffa-b7ca3ac38d51/volumes/kubernetes.io~projected/kube-api-access-z97l8:{mountpoint:/var/lib/kubelet/pods/800f854c-c40c-45d9-9ffa-b7ca3ac38d51/volumes/kubernetes.io~projected/kube-api-access-z97l8 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e5ac899-ecba-4c42-aa7a-e34e14269efa/volumes/kubernetes.io~projected/kube-api-access-lnnxf:{mountpoint:/var/lib/kubelet/pods/9e5ac899-ecba-4c42-aa7a-e34e14269efa/volumes/kubernetes.io~projected/kube-api-access-lnnxf major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e9032e7-1ec1-4a5d-8e28-622d4fd01392/volumes/kubernetes.io~projected/kube-api-access-pgfnf:{mountpoint:/var/lib/kubelet/pods/9e9032e7-1ec1-4a5d-8e28-622d4fd01392/volumes/kubernetes.io~projected/kube-api-access-pgfnf major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a7be31dd-dd7b-448e-938b-ba7a0807bf49/volumes/kubernetes.io~projected/kube-api-access-w86dj:{mountpoint:/var/lib/kubelet/pods/a7be31dd-dd7b-448e-938b-ba7a0807bf49/volumes/kubernetes.io~projected/kube-api-access-w86dj major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3/volumes/kubernetes.io~projected/kube-api-access-7tc4s:{mountpoint:/var/lib/kubelet/pods/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3/volumes/kubernetes.io~projected/kube-api-access-7tc4s major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac85734e-3f97-473f-b3ab-8aadd898f381/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/ac85734e-3f97-473f-b3ab-8aadd898f381/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae2cac8d-509e-4e22-b35c-ab6d662fdf24/volumes/kubernetes.io~projected/kube-api-access-nmgk4:{mountpoint:/var/lib/kubelet/pods/ae2cac8d-509e-4e22-b35c-ab6d662fdf24/volumes/kubernetes.io~projected/kube-api-access-nmgk4 major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b0939980-01bb-4ae3-9c95-5e190b719cbe/volumes/kubernetes.io~projected/kube-api-access-jd89g:{mountpoint:/var/lib/kubelet/pods/b0939980-01bb-4ae3-9c95-5e190b719cbe/volumes/kubernetes.io~projected/kube-api-access-jd89g major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes/kubernetes.io~projected/kube-api-access-p9mdl:{mountpoint:/var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes/kubernetes.io~projected/kube-api-access-p9mdl major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d2b52b9e-4797-4a2e-953e-7da0f61bca87/volumes/kubernetes.io~projected/kube-api-access-97wlp:{mountpoint:/var/lib/kubelet/pods/d2b52b9e-4797-4a2e-953e-7da0f61bca87/volumes/kubernetes.io~projected/kube-api-access-97wlp major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d2b52b9e-4797-4a2e-953e-7da0f61bca87/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/d2b52b9e-4797-4a2e-953e-7da0f61bca87/volumes/kubernetes.io~secret/serving-cert major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes/kubernetes.io~projected/kube-api-access-574hk:{mountpoint:/var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes/kubernetes.io~projected/kube-api-access-574hk major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e7b7aaa9-68a9-4259-a9ce-001217dfa67f/volumes/kubernetes.io~projected/kube-api-access-26kg9:{mountpoint:/var/lib/kubelet/pods/e7b7aaa9-68a9-4259-a9ce-001217dfa67f/volumes/kubernetes.io~projected/kube-api-access-26kg9 major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/edd196c7-d883-4462-b422-385570ed9169/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/edd196c7-d883-4462-b422-385570ed9169/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/edd196c7-d883-4462-b422-385570ed9169/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/edd196c7-d883-4462-b422-385570ed9169/volumes/kubernetes.io~empty-dir/tmp major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/edd196c7-d883-4462-b422-385570ed9169/volumes/kubernetes.io~projected/kube-api-access-tqjpj:{mountpoint:/var/lib/kubelet/pods/edd196c7-d883-4462-b422-385570ed9169/volumes/kubernetes.io~projected/kube-api-access-tqjpj major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f0ba138d-0a24-4f77-b1f4-c0861a48797f/volumes/kubernetes.io~projected/kube-api-access-t5vxf:{mountpoint:/var/lib/kubelet/pods/f0ba138d-0a24-4f77-b1f4-c0861a48797f/volumes/kubernetes.io~projected/kube-api-access-t5vxf major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f16c549e-41e6-4721-ab5f-91b37f05a97c/volumes/kubernetes.io~projected/kube-api-access-vxx6s:{mountpoint:/var/lib/kubelet/pods/f16c549e-41e6-4721-ab5f-91b37f05a97c/volumes/kubernetes.io~projected/kube-api-access-vxx6s major:0 minor:205 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/cd7f62035a8243694e1c7c46aa5e3ca5e04d4b00e4176a0ff01a5e3495b81f1b/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/59fd8c146862e1138a92be6b34248d102f5e298ecad1549b88ab662c99b38df3/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/6c25900edcaf2aab4daffdebb9e646ca536e64fbecf0aa14be945b39cee5b2c6/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/2fbe86187bf39cc9135cb68265f47f3b149489ddc46d1ade044b15a64a562a54/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/f9859b99e41bda6f5f307295df493d8392aa0e796b455aa56b9f9014470f2d7d/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/031fe0d7116a5cfd58c072060d8af38ced1da294dcee8d96cf72ead051285bf9/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/6d7ac5265287ca66e4f1d1a573ee8438194eb3e89bc40a4a5d6ab877a6eb7e44/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/e90fac60ee4739299c5228a017417abe4647920347bdd135595e083b1749e987/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/ddf79b468ff0a3255b674dd898985c5a852c73b409509275702509809fdccef1/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/780fb646a7c35c1972af6632e744eb72327b821ae42d9ceeb45560e354b273ef/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/e9894ddfe934e4ac6d40dd0e5bb454e6b8d255ecaef5c5926d3a789692718421/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-145:{mountpoint:/var/lib/containers/storage/overlay/b697b427cf296b86c0c64fee794ce4d067cbdd8c7eec1bc066e498a68214a0f0/merged major:0 minor:145 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/f508d30ef484a64f338dea1581ff564740818b00a942973698176ce945fa4d69/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/77655170336f79ad6b05dd94d7d7a411ebef2125978ff89d375b3f637527f1e2/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/68290624f8991da7e9938815fb5c0e457aac71e3b13df7be2012a393b6d64df1/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/124d8e32226c6efbf1baeea13ad3e67a7aa30757631768a41e1a98d99a29a44e/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/857a3e4899ef23f2f81d2657bfa64d89253cc45dc943da4720f2d2e2b3fc27fd/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d0b22e29d3a060061d7d95258bf6487d718ad8de70864541e2acfb8bcd7b97bc/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/97d3291b768e4efaf434fbed1f9e06291e4361f8a37e7e29b2c8c853bbe59af5/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-184:{mountpoint:/var/lib/containers/storage/overlay/d259ac804c19b10842f3c8224beeef0f4f1d556ad2b03ff57d015689e563e622/merged major:0 minor:184 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/7b730bcc71f877c360c9213a57b28667abe8bca142b9fe1c4693544f6aab7813/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/90e8f51606c678c1f850116849a2acc59171f454622d03559a5e5e0e17497980/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/f4582d5fb6acae5110e4ab16cf96b63fc404712bda5044299defdb766c74c843/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/b147e19b2db2fe469008ae2df03a3ea473713a9fe6d845f7ab8d3da26d612f55/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/0a6ba2c016cdef332a1691e574105730fca22e6b3307eb03b28c396a979fdaba/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/eacd52f9530af8e70aacd08582be8fdb885f25f016b65dbc78a65d6f315a5f15/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/0469b004eccfe1f2d2071aa459676b68ba8031d96abb54117c76992a5831dbe4/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/c5c75132bdcced68329eccc0a4152c3af4e1813f44ee21596f8232328e190da3/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/7822177603de33ea813bcd00c76c9fcf6c6e65aae43c2834ea13a38b18b8cdd0/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/b20e679f6effcd4f4ee4e8c2a25fc9067de469a5d05b811a5bfdc033d6735b8c/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/c552439b9bf94688d5dca3050f745c18a938168f3d5826ed419605d5cd53fc85/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-275:{mountpoint:/var/lib/containers/storage/overlay/bac7f9c1e8c7e61f945a06c9b4a7fe15061ab61e8d8fb73a7c2146dd62651ef6/merged major:0 minor:275 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/aad7535ea87a32637e4c55e67efb602d1d831bcd3be8dbc083573eecfc4f5556/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/6ee21b4ba9253ea6c89fa270ae5c95baac46c13521c15f57c8ec809a23a709b8/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-281:{mountpoint:/var/lib/containers/storage/overlay/910dc72042398f0c4a03a398f92262d99e0a97d50b244afd4368309aa3ad1664/merged major:0 minor:281 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/7890e27ed13da32922acf68ec9adb1c37b799fcb4a48cdc468034ffe4166e901/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/0cf1f52eaa42202f185c8c581b97fbd224597c17a7ed23af3f76f79e22b16640/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/a433d6da2a88ae20232cb20fd70e4fe7b95adaabb6a63864b71ce764c994ba0f/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/1c49c8780282bcb92a272681140a526e25e9191237d9b4ca8eb7451011da1d0d/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/14df356522891a0a4ed6a2f041c1715262657a4e5a2d699d3bd8b14118eb750a/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/6e2d2368410d54321c562ca2ac5d2b6ef71412083a3ce7c04d50700682db1ac4/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-318:{mountpoint:/var/lib/containers/storage/overlay/d45adfd599f41c27546b3f4a2b25d0a8099478a0cebf323595312e1893325fbc/merged major:0 minor:318 fsType:overlay blockSize:0} overlay_0-324:{mountpoint:/var/lib/containers/storage/overlay/69907e5a5df8787c26db89358fe689cfacd62dc85435fc318840e9dc462a5288/merged major:0 minor:324 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/ac23eeb3b3c4244a61975adffe65a2af633538e6cc9922e0197b5db63afab97b/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/3a813eff8e12e0f410b0be781c0600a7dd539a0913e0e9ef62e6cedb51f313ba/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/1dab1aba596a4a9842c3af3c59be2533c63d3d4579f4f35b20a936c23d9af40e/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/887e7b26b9283720e971c288d339c14cec224792e5b18a05200201645021c6d2/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/d67033be27cf34203aeb1bb0c198d6ac0d6efd73112decdace996de3d7313b1a/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/a5446a3870c17f001c2a1383fb2cb3470dd41986a925bb79a0cc45d023e036f2/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/19197ae6bc335f23d8d2df34a9dc71ba136fe7400b3df7c02ef09b8af7b4e155/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/c66483e8f14b66b444b7d6ceb818b690a258bf5cf397269db19653de3ebec49d/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/1dc26ef2170c0514a03b81c96e9c0e665373a2905c0bfb553bdf889826c90c32/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/bf3ee1c9ef28702e2d4b66d78d315bf86ef0ec96f28c1c28e42018f4d56738cf/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/d22c6b9abb7d65895b6bcaf2358fe82bb8568e58a60a5ab4e2db631597e8d73e/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/1ebfd4d336a3ad8b2e4682a8fa863d0e51ed92de9c0b3ad4e6e06c1595f0a997/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/84d3923dec1738f91dd91371e7f800f12dee43f4dee421ba8334c67b67346aef/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.364528    4925 manager.go:217] Machine: {Timestamp:2026-10-30 18:36:18.363503317 +0000 UTC m=+0.079848384 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098705 MemoryCapacity:32812011520 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec286d0868f22185312742c111bbe479 SystemUUID:ec286d08-68f2-2185-3127-42c111bbe479 BootID:7050a5f8-ade0-4f7f-bcae-c3d88b552adf Filesystems:[{Device:/var/lib/kubelet/pods/3762f795-944b-458e-9458-c26eaf133871/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fda3295ed6547c21104bce3ea46135cc2b26d2e328d9d58fca1a66beea3b5b97/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3e7518c9-9163-41ac-9546-556b1773cec6/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/f0ba138d-0a24-4f77-b1f4-c0861a48797f/volumes/kubernetes.io~projected/kube-api-access-t5vxf DeviceMajor:0 DeviceMinor:223 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ac85734e-3f97-473f-b3ab-8aadd898f381/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:288 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/edd196c7-d883-4462-b422-385570ed9169/volumes/kubernetes.io~projected/kube-api-access-tqjpj DeviceMajor:0 DeviceMinor:67 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:197 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/9e5ac899-ecba-4c42-aa7a-e34e14269efa/volumes/kubernetes.io~projected/kube-api-access-lnnxf DeviceMajor:0 DeviceMinor:216 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c4f431637bd8ea72fd98b092b53df10b00f0d0a3ba3a856b957e0501a506bece/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4c1460ce-789a-411c-b748-2b3506811aa1/volumes/kubernetes.io~projected/kube-api-access-kpvfc DeviceMajor:0 DeviceMinor:57 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/05882ddb5a0f5813f31f6de21cef09f1ccbd8629b246f700e925c91beb5340f0/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/ae2cac8d-509e-4e22-b35c-ab6d662fdf24/volumes/kubernetes.io~projected/kube-api-access-nmgk4 DeviceMajor:0 DeviceMinor:204 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d2b52b9e-4797-4a2e-953e-7da0f61bca87/volumes/kubernetes.io~projected/kube-api-access-97wlp DeviceMajor:0 DeviceMinor:217 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d34814233153f704c3a04db0cd80b7994903b3e1dc582de690be12869ca63296/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/628d4dcc-30b9-4817-815e-53d36818b7c2/volumes/kubernetes.io~projected/kube-api-access-k7bxj DeviceMajor:0 DeviceMinor:59 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-318 DeviceMajor:0 DeviceMinor:318 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/faa1dd80e2e92bf9d05e3aea03b5bd5733995ad7888281b0af8b4079958531c0/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/800f854c-c40c-45d9-9ffa-b7ca3ac38d51/volumes/kubernetes.io~projected/kube-api-access-z97l8 DeviceMajor:0 DeviceMinor:56 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c3fe00b3d12767cf8289d90c5eaeae0d214b23f5e82fb05d7a22cbd0da0cf4de/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3/volumes/kubernetes.io~projected/kube-api-access-7tc4s DeviceMajor:0 DeviceMinor:58 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/3e7518c9-9163-41ac-9546-556b1773cec6/volumes/kubernetes.io~projected/kube-api-access-vgqmm DeviceMajor:0 DeviceMinor:66 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/272efc3d-7980-4fd1-a20d-9bd023f48bec/volumes/kubernetes.io~projected/kube-api-access-dkr9h DeviceMajor:0 DeviceMinor:222 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6ab635073ce18d9860646ba15f6fd95ea3ec30b03f52d9724794589a435d01b9/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-184 DeviceMajor:0 DeviceMinor:184 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9e9032e7-1ec1-4a5d-8e28-622d4fd01392/volumes/kubernetes.io~projected/kube-api-access-pgfnf DeviceMajor:0 DeviceMinor:219 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5a42aedc4ebe3ce8adbed1030f1b5994ef0e8cc402352acf984b72350f25a3b1/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2c77e172-a80d-4eda-986d-ec8c3c52e131/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:210 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7e71a6838390fe0ef8b0685088c12fec304c859abebc15e65eb0ef7a0f650886/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-275 DeviceMajor:0 DeviceMinor:275 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-281 DeviceMajor:0 DeviceMinor:281 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f16c549e-41e6-4721-ab5f-91b37f05a97c/volumes/kubernetes.io~projected/kube-api-access-vxx6s DeviceMajor:0 DeviceMinor:205 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/0b90705d-6abb-481d-8af0-89e542709329/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:208 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5886c3667fcb917c58b17d5a153971827aea2790497f54c1a53f6d71ad31dd84/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4bee5094-7886-4fa9-aa7c-0ddc3d518619/volumes/kubernetes.io~projected/kube-api-access-84gss DeviceMajor:0 DeviceMinor:224 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f35a1bc16f4abc1d8257b702bc5da0459be08a98e1f4ff005bd0693db0140d57/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/518b6b63e2b339f44b2facd01785314e1b0649197ae8ea149933195c63c6821c/userdata/shm DeviceMajor:0 DeviceMinor:237 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8ebf01bb27e75d8f710025340685c56c55877a45e266253db115f25358a5b0de/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/edd196c7-d883-4462-b422-385570ed9169/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:62 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c45e8b0359365cc553682db8d420ee0da63380c7193b24dcf96d959336d76970/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9044b821157b9dd8b87564d99f0be80e50cabac7d4c89cb19da361f37738104c/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/abd8ee6926056d9e202f1f341f371a7d577561fc4fc8005f5652ff22ac4fb453/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/5cf855c4-4e7e-4669-9adc-583e22e27e73/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:215 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:226 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eb46efb03bca9a23e02ad293848408bc0d72301fe54b8942d15b471d323bc9b5/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7563c219-088f-4c66-81f5-86c581f08f34/volumes/kubernetes.io~projected/kube-api-access-ztnjp DeviceMajor:0 DeviceMinor:61 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:213 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c21c20d181f19eb90e9a3db0649f647bd4f23ceeaf4fd9e4a9bdde086bb18043/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d2b52b9e-4797-4a2e-953e-7da0f61bca87/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:211 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/272efc3d-7980-4fd1-a20d-9bd023f48bec/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:214 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-324 DeviceMajor:0 DeviceMinor:324 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/82bd5c28c18ae081117ba6aeb96079c9aa67f6fea2f0dc9e057338aa6d805c33/userdata/shm DeviceMajor:0 DeviceMinor:328 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e7aab5272fcd7c97b395509f63b18d1012b5eec1aeae7c72965a6bce0c3089ae/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b0939980-01bb-4ae3-9c95-5e190b719cbe/volumes/kubernetes.io~projected/kube-api-access-jd89g DeviceMajor:0 DeviceMinor:201 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2c77e172-a80d-4eda-986d-ec8c3c52e131/volumes/kubernetes.io~projected/kube-api-access-f9mtz DeviceMajor:0 DeviceMinor:227 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/edd196c7-d883-4462-b422-385570ed9169/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:64 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/0b90705d-6abb-481d-8af0-89e542709329/volumes/kubernetes.io~projected/kube-api-access-7dn5r DeviceMajor:0 DeviceMinor:218 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes/kubernetes.io~projected/kube-api-access-p9mdl DeviceMajor:0 DeviceMinor:220 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a7be31dd-dd7b-448e-938b-ba7a0807bf49/volumes/kubernetes.io~projected/kube-api-access-w86dj DeviceMajor:0 DeviceMinor:293 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3e7518c9-9163-41ac-9546-556b1773cec6/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:63 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/5cf855c4-4e7e-4669-9adc-583e22e27e73/volumes/kubernetes.io~projected/kube-api-access-tl4tg DeviceMajor:0 DeviceMinor:221 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/70da42f1-d38f-490e-8db0-4c1d141b2009/volumes/kubernetes.io~projected/kube-api-access-79swd DeviceMajor:0 DeviceMinor:326 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:209 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6c97f9bda22eeb02b88ccb7e65301e762ba929e0195163087eccf0d92da587e5/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fd3097b0bdebb315f864da8d33792c64b4a2f02544f222a4070b80d6093782d6/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e7b7aaa9-68a9-4259-a9ce-001217dfa67f/volumes/kubernetes.io~projected/kube-api-access-26kg9 DeviceMajor:0 DeviceMinor:327 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:45 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/4cd65688-51e1-4b8c-8dc5-6e3771a74e66/volumes/kubernetes.io~projected/kube-api-access-jms4w DeviceMajor:0 DeviceMinor:206 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/4bee5094-7886-4fa9-aa7c-0ddc3d518619/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:212 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/734db718-c901-4fe7-a8e1-b90823ccd66c/volumes/kubernetes.io~projected/kube-api-access-s9q7l DeviceMajor:0 DeviceMinor:225 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8aed94ba983787ab53fa85bf2c419096c9b05b013a9bbd1746f293ec8216d2d0/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-145 DeviceMajor:0 DeviceMinor:145 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/74a6aabd-b72d-422d-ac40-7d07a9e56a41/volumes/kubernetes.io~projected/kube-api-access-z9ndr DeviceMajor:0 DeviceMinor:65 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:202 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/2c77e172-a80d-4eda-986d-ec8c3c52e131/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:207 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/36d759c0-ed6e-4170-aa8f-70560074a467/volumes/kubernetes.io~projected/kube-api-access-6fn8p DeviceMajor:0 DeviceMinor:60 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes/kubernetes.io~projected/kube-api-access-574hk DeviceMajor:0 DeviceMinor:203 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:518b6b63e2b339f MacAddress:da:85:d8:26:5e:0f Speed:10000 Mtu:8901} {Name:5886c3667fcb917 MacAddress:e2:4f:53:d7:37:26 Speed:10000 Mtu:8901} {Name:5a42aedc4ebe3ce MacAddress:36:f4:1d:d2:43:ab Speed:10000 Mtu:8901} {Name:6ab635073ce18d9 MacAddress:f2:29:ef:c0:27:66 Speed:10000 Mtu:8901} {Name:6c97f9bda22eeb0 MacAddress:96:dc:c0:f3:48:1c Speed:10000 Mtu:8901} {Name:82bd5c28c18ae08 MacAddress:e2:2e:f7:ba:19:3e Speed:10000 Mtu:8901} {Name:8ebf01bb27e75d8 MacAddress:06:5a:e1:49:bb:3d Speed:10000 Mtu:8901} {Name:abd8ee6926056d9 MacAddress:9e:4b:34:da:bb:78 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:f2:64:72:fb Speed:0 Mtu:9001} {Name:br-int MacAddress:32:4d:49:7a:65:bc Speed:0 Mtu:8901} {Name:c4f431637bd8ea7 MacAddress:ca:1d:5c:9a:e3:58 Speed:10000 Mtu:8901} {Name:d34814233153f70 MacAddress:16:4b:8d:33:b0:56 Speed:10000 Mtu:8901} {Name:eb46efb03bca9a2 MacAddress:22:99:98:c8:03:36 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:f2:64:72:fb Speed:0 Mtu:9001} {Name:f35a1bc16f4abc1 MacAddress:fe:f2:8c:57:3c:e4 Speed:10000 Mtu:8901} {Name:fd3097b0bdebb31 MacAddress:4e:49:bb:d0:72:4a Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:16:b3:8b:0f:8e:dc Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ba:0e:4b:37:5b:30 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812011520 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365172    4925 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365229    4925 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365503    4925 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365529    4925 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-58.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365644    4925 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365652    4925 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365661    4925 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365674    4925 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365691    4925 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365831    4925 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365925    4925 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365957    4925 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365967    4925 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365975    4925 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365985    4925 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.365992    4925 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.366707    4925 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.366717    4925 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:36:18.368797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.367138    4925 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369256    4925 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369265    4925 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369365    4925 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369543    4925 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369569    4925 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369677    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369690    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369696    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369702    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369709    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369718    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369727    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369734    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369740    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369746    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369762    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369772    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369795    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:36:18.369930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.369801    4925 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:36:18.373860 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.373841    4925 server.go:1263] "Started kubelet"
Oct 30 18:36:18.373957 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.373891    4925 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:36:18.374036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.373984    4925 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:36:18.374109 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.374067    4925 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:36:18.374417 ip-10-0-134-58 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:36:18.374605 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.374466    4925 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:36:18.377181 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.376993    4925 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:36:18.381302 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.381278    4925 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:18.382354 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.382337    4925 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:18.383268 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.383244    4925 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:36:18.383357 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.383343    4925 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:36:18.383357 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.383352    4925 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:36:18.383456 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.383254    4925 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:36:18.383501 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.383471    4925 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:36:18.383566 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.383541    4925 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:30:39 +0000 UTC" deadline="2028-05-21 05:43:13.200547419 +0000 UTC"
Oct 30 18:36:18.383623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.383565    4925 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13643h6m54.816985283s"
Oct 30 18:36:18.384250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.384227    4925 factory.go:55] Registering systemd factory
Oct 30 18:36:18.384342 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.384253    4925 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.384460    4925 factory.go:153] Registering CRI-O factory
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.384478    4925 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.384540    4925 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.385184    4925 factory.go:103] Registering Raw factory
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.385217    4925 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.385643    4925 manager.go:319] Starting recovery of all containers
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.386906    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b90705d-6abb-481d-8af0-89e542709329" volumeName="kubernetes.io/secret/0b90705d-6abb-481d-8af0-89e542709329-serving-cert" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.386941    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="628d4dcc-30b9-4817-815e-53d36818b7c2" volumeName="kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.386955    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="800f854c-c40c-45d9-9ffa-b7ca3ac38d51" volumeName="kubernetes.io/configmap/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-iptables-alerter-script" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387008    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3" volumeName="kubernetes.io/projected/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-kube-api-access-7tc4s" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387022    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac85734e-3f97-473f-b3ab-8aadd898f381" volumeName="kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387034    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="36d759c0-ed6e-4170-aa8f-70560074a467" volumeName="kubernetes.io/configmap/36d759c0-ed6e-4170-aa8f-70560074a467-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387521    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e7518c9-9163-41ac-9546-556b1773cec6" volumeName="kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-env-overrides" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387537    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="628d4dcc-30b9-4817-815e-53d36818b7c2" volumeName="kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387551    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d2b52b9e-4797-4a2e-953e-7da0f61bca87" volumeName="kubernetes.io/projected/d2b52b9e-4797-4a2e-953e-7da0f61bca87-kube-api-access-97wlp" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387588    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e32eb376-a16f-421c-8958-a410a5650964" volumeName="kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-registry-certificates" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387601    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b90705d-6abb-481d-8af0-89e542709329" volumeName="kubernetes.io/configmap/0b90705d-6abb-481d-8af0-89e542709329-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387616    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="36d759c0-ed6e-4170-aa8f-70560074a467" volumeName="kubernetes.io/projected/36d759c0-ed6e-4170-aa8f-70560074a467-kube-api-access-6fn8p" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387628    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4c1460ce-789a-411c-b748-2b3506811aa1" volumeName="kubernetes.io/projected/4c1460ce-789a-411c-b748-2b3506811aa1-kube-api-access-kpvfc" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387659    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cf855c4-4e7e-4669-9adc-583e22e27e73" volumeName="kubernetes.io/configmap/5cf855c4-4e7e-4669-9adc-583e22e27e73-trusted-ca" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387675    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74a6aabd-b72d-422d-ac40-7d07a9e56a41" volumeName="kubernetes.io/projected/74a6aabd-b72d-422d-ac40-7d07a9e56a41-kube-api-access-z9ndr" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387709    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a7be31dd-dd7b-448e-938b-ba7a0807bf49" volumeName="kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387725    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c77e172-a80d-4eda-986d-ec8c3c52e131" volumeName="kubernetes.io/projected/2c77e172-a80d-4eda-986d-ec8c3c52e131-kube-api-access-f9mtz" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387738    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="36d759c0-ed6e-4170-aa8f-70560074a467" volumeName="kubernetes.io/configmap/36d759c0-ed6e-4170-aa8f-70560074a467-multus-daemon-config" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387751    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4cd65688-51e1-4b8c-8dc5-6e3771a74e66" volumeName="kubernetes.io/projected/4cd65688-51e1-4b8c-8dc5-6e3771a74e66-kube-api-access-jms4w" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387764    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="734db718-c901-4fe7-a8e1-b90823ccd66c" volumeName="kubernetes.io/projected/734db718-c901-4fe7-a8e1-b90823ccd66c-kube-api-access-s9q7l" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387776    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e9032e7-1ec1-4a5d-8e28-622d4fd01392" volumeName="kubernetes.io/configmap/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-config" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387792    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e32eb376-a16f-421c-8958-a410a5650964" volumeName="kubernetes.io/empty-dir/e32eb376-a16f-421c-8958-a410a5650964-ca-trust-extracted" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387804    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e32eb376-a16f-421c-8958-a410a5650964" volumeName="kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-bound-sa-token" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387818    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d2b52b9e-4797-4a2e-953e-7da0f61bca87" volumeName="kubernetes.io/configmap/d2b52b9e-4797-4a2e-953e-7da0f61bca87-config" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387829    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edd196c7-d883-4462-b422-385570ed9169" volumeName="kubernetes.io/empty-dir/edd196c7-d883-4462-b422-385570ed9169-etc-tuned" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387842    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b90705d-6abb-481d-8af0-89e542709329" volumeName="kubernetes.io/empty-dir/0b90705d-6abb-481d-8af0-89e542709329-tmp" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387858    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="272efc3d-7980-4fd1-a20d-9bd023f48bec" volumeName="kubernetes.io/projected/272efc3d-7980-4fd1-a20d-9bd023f48bec-kube-api-access-dkr9h" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387871    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7563c219-088f-4c66-81f5-86c581f08f34" volumeName="kubernetes.io/empty-dir/7563c219-088f-4c66-81f5-86c581f08f34-tmp-dir" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387904    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd26f706-5971-4c47-a63e-699e7962ef8e" volumeName="kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-kube-api-access-p9mdl" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387921    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd26f706-5971-4c47-a63e-699e7962ef8e" volumeName="kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-installation-pull-secrets" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387933    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e32eb376-a16f-421c-8958-a410a5650964" volumeName="kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-trusted-ca" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387947    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e32eb376-a16f-421c-8958-a410a5650964" volumeName="kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387961    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bee5094-7886-4fa9-aa7c-0ddc3d518619" volumeName="kubernetes.io/configmap/4bee5094-7886-4fa9-aa7c-0ddc3d518619-config" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387975    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7b7aaa9-68a9-4259-a9ce-001217dfa67f" volumeName="kubernetes.io/empty-dir/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-data-volume" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.387989    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0ba138d-0a24-4f77-b1f4-c0861a48797f" volumeName="kubernetes.io/configmap/f0ba138d-0a24-4f77-b1f4-c0861a48797f-telemetry-config" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388001    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0ba138d-0a24-4f77-b1f4-c0861a48797f" volumeName="kubernetes.io/projected/f0ba138d-0a24-4f77-b1f4-c0861a48797f-kube-api-access-t5vxf" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388014    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b0939980-01bb-4ae3-9c95-5e190b719cbe" volumeName="kubernetes.io/projected/b0939980-01bb-4ae3-9c95-5e190b719cbe-kube-api-access-jd89g" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388027    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b90705d-6abb-481d-8af0-89e542709329" volumeName="kubernetes.io/projected/0b90705d-6abb-481d-8af0-89e542709329-kube-api-access-7dn5r" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388126    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="272efc3d-7980-4fd1-a20d-9bd023f48bec" volumeName="kubernetes.io/secret/272efc3d-7980-4fd1-a20d-9bd023f48bec-cert" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388143    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bee5094-7886-4fa9-aa7c-0ddc3d518619" volumeName="kubernetes.io/projected/4bee5094-7886-4fa9-aa7c-0ddc3d518619-kube-api-access-84gss" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388160    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e5ac899-ecba-4c42-aa7a-e34e14269efa" volumeName="kubernetes.io/projected/9e5ac899-ecba-4c42-aa7a-e34e14269efa-kube-api-access-lnnxf" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388176    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3" volumeName="kubernetes.io/configmap/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-serviceca" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388188    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c77e172-a80d-4eda-986d-ec8c3c52e131" volumeName="kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-stats-auth" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388205    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4bee5094-7886-4fa9-aa7c-0ddc3d518619" volumeName="kubernetes.io/secret/4bee5094-7886-4fa9-aa7c-0ddc3d518619-serving-cert" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388236    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cf855c4-4e7e-4669-9adc-583e22e27e73" volumeName="kubernetes.io/secret/5cf855c4-4e7e-4669-9adc-583e22e27e73-serving-cert" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388248    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="628d4dcc-30b9-4817-815e-53d36818b7c2" volumeName="kubernetes.io/projected/628d4dcc-30b9-4817-815e-53d36818b7c2-kube-api-access-k7bxj" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388262    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="734db718-c901-4fe7-a8e1-b90823ccd66c" volumeName="kubernetes.io/configmap/734db718-c901-4fe7-a8e1-b90823ccd66c-config-volume" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388275    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd26f706-5971-4c47-a63e-699e7962ef8e" volumeName="kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388312    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2c77e172-a80d-4eda-986d-ec8c3c52e131" volumeName="kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-default-certificate" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388341    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="800f854c-c40c-45d9-9ffa-b7ca3ac38d51" volumeName="kubernetes.io/projected/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-kube-api-access-z97l8" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388355    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd26f706-5971-4c47-a63e-699e7962ef8e" volumeName="kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-trusted-ca" seLinuxMountContext=""
Oct 30 18:36:18.388330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388369    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7b7aaa9-68a9-4259-a9ce-001217dfa67f" volumeName="kubernetes.io/projected/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-api-access-26kg9" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388383    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edd196c7-d883-4462-b422-385570ed9169" volumeName="kubernetes.io/projected/edd196c7-d883-4462-b422-385570ed9169-kube-api-access-tqjpj" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388395    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3762f795-944b-458e-9458-c26eaf133871" volumeName="kubernetes.io/configmap/3762f795-944b-458e-9458-c26eaf133871-konnectivity-ca" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388408    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f478aac2-de91-4b13-836b-ba09809222f0" volumeName="kubernetes.io/configmap/f478aac2-de91-4b13-836b-ba09809222f0-nginx-conf" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388419    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b90705d-6abb-481d-8af0-89e542709329" volumeName="kubernetes.io/configmap/0b90705d-6abb-481d-8af0-89e542709329-service-ca-bundle" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388439    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3762f795-944b-458e-9458-c26eaf133871" volumeName="kubernetes.io/secret/3762f795-944b-458e-9458-c26eaf133871-agent-certs" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388450    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cf855c4-4e7e-4669-9adc-583e22e27e73" volumeName="kubernetes.io/projected/5cf855c4-4e7e-4669-9adc-583e22e27e73-kube-api-access-tl4tg" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388462    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70da42f1-d38f-490e-8db0-4c1d141b2009" volumeName="kubernetes.io/projected/70da42f1-d38f-490e-8db0-4c1d141b2009-kube-api-access-79swd" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388474    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7563c219-088f-4c66-81f5-86c581f08f34" volumeName="kubernetes.io/projected/7563c219-088f-4c66-81f5-86c581f08f34-kube-api-access-ztnjp" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388532    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="edd196c7-d883-4462-b422-385570ed9169" volumeName="kubernetes.io/empty-dir/edd196c7-d883-4462-b422-385570ed9169-tmp" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388546    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f16c549e-41e6-4721-ab5f-91b37f05a97c" volumeName="kubernetes.io/projected/f16c549e-41e6-4721-ab5f-91b37f05a97c-kube-api-access-vxx6s" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388558    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b90705d-6abb-481d-8af0-89e542709329" volumeName="kubernetes.io/empty-dir/0b90705d-6abb-481d-8af0-89e542709329-snapshots" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388570    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd26f706-5971-4c47-a63e-699e7962ef8e" volumeName="kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-certificates" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388604    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd26f706-5971-4c47-a63e-699e7962ef8e" volumeName="kubernetes.io/empty-dir/cd26f706-5971-4c47-a63e-699e7962ef8e-ca-trust-extracted" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388615    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e32eb376-a16f-421c-8958-a410a5650964" volumeName="kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-installation-pull-secrets" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388626    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e7518c9-9163-41ac-9546-556b1773cec6" volumeName="kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388641    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e7518c9-9163-41ac-9546-556b1773cec6" volumeName="kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-ovnkube-config" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388653    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e7518c9-9163-41ac-9546-556b1773cec6" volumeName="kubernetes.io/secret/3e7518c9-9163-41ac-9546-556b1773cec6-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388665    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cf855c4-4e7e-4669-9adc-583e22e27e73" volumeName="kubernetes.io/configmap/5cf855c4-4e7e-4669-9adc-583e22e27e73-config" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388676    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="628d4dcc-30b9-4817-815e-53d36818b7c2" volumeName="kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388697    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="734db718-c901-4fe7-a8e1-b90823ccd66c" volumeName="kubernetes.io/empty-dir/734db718-c901-4fe7-a8e1-b90823ccd66c-tmp-dir" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388724    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd26f706-5971-4c47-a63e-699e7962ef8e" volumeName="kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-bound-sa-token" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388737    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d2b52b9e-4797-4a2e-953e-7da0f61bca87" volumeName="kubernetes.io/secret/d2b52b9e-4797-4a2e-953e-7da0f61bca87-serving-cert" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388748    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e7518c9-9163-41ac-9546-556b1773cec6" volumeName="kubernetes.io/projected/3e7518c9-9163-41ac-9546-556b1773cec6-kube-api-access-vgqmm" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388759    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e9032e7-1ec1-4a5d-8e28-622d4fd01392" volumeName="kubernetes.io/projected/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-kube-api-access-pgfnf" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388773    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae2cac8d-509e-4e22-b35c-ab6d662fdf24" volumeName="kubernetes.io/projected/ae2cac8d-509e-4e22-b35c-ab6d662fdf24-kube-api-access-nmgk4" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388785    4925 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e32eb376-a16f-421c-8958-a410a5650964" volumeName="kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-kube-api-access-574hk" seLinuxMountContext=""
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388796    4925 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:36:18.391238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.388822    4925 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:36:18.394788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.393688    4925 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:18.394788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.394067    4925 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 18:36:18.400253 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.400231    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/1.log"
Oct 30 18:36:18.401000 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.400980    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/1.log"
Oct 30 18:36:18.412188 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.412160    4925 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:36:18.413891 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.413860    4925 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:36:18.413891 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.413882    4925 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:36:18.414037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.413902    4925 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:36:18.414037 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:18.413965    4925 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 18:36:18.417366 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.417338    4925 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:18.438257 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.438237    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/1.log"
Oct 30 18:36:18.438356 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.438266    4925 generic.go:356] "Generic (PLEG): container finished" podID="5cf855c4-4e7e-4669-9adc-583e22e27e73" containerID="70c2560feead44db5071af9e4bce3fdbad88ffcafac5d9145f84a398128d94bb" exitCode=255
Oct 30 18:36:18.444518 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.444495    4925 generic.go:356] "Generic (PLEG): container finished" podID="f3cd2cf92111a4cf6f6c288eae5fe1a5" containerID="c5693126742fbfe3c07fa4c84b9b521b2c4215309736ab1c2fc8bb7010202e48" exitCode=0
Oct 30 18:36:18.461368 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.461342    4925 generic.go:356] "Generic (PLEG): container finished" podID="628d4dcc-30b9-4817-815e-53d36818b7c2" containerID="39b150b284673cfc3fc7e791833a88b09189820c03a7ed795f8d4b0c6b60f8ff" exitCode=0
Oct 30 18:36:18.461447 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.461370    4925 generic.go:356] "Generic (PLEG): container finished" podID="628d4dcc-30b9-4817-815e-53d36818b7c2" containerID="f05cc31a335413ada85cb1b294e5aa9025f681cab032e0ed28c885249900378e" exitCode=0
Oct 30 18:36:18.461447 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.461381    4925 generic.go:356] "Generic (PLEG): container finished" podID="628d4dcc-30b9-4817-815e-53d36818b7c2" containerID="24ccaaac8c0e953005ccba478e23eef967fbfd3df6b26c8f9cecaba817043fd0" exitCode=0
Oct 30 18:36:18.461447 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.461388    4925 generic.go:356] "Generic (PLEG): container finished" podID="628d4dcc-30b9-4817-815e-53d36818b7c2" containerID="5bc83cb525d355860c03050a55f5a5ef5cee7c6198b2c588f21b84489b107f57" exitCode=0
Oct 30 18:36:18.461447 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.461398    4925 generic.go:356] "Generic (PLEG): container finished" podID="628d4dcc-30b9-4817-815e-53d36818b7c2" containerID="e6dc8799f37c897422563741be9ca838a52d5411d873a163ed2f7791671ff8c1" exitCode=0
Oct 30 18:36:18.461447 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.461410    4925 generic.go:356] "Generic (PLEG): container finished" podID="628d4dcc-30b9-4817-815e-53d36818b7c2" containerID="bdf711f80f0dfaf196b2d41346fe428f6be0947244a5357a377f0059fffd35e6" exitCode=0
Oct 30 18:36:18.475467 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.475453    4925 manager.go:324] Recovery completed
Oct 30 18:36:18.494484 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494469    4925 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:36:18.494484 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494482    4925 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:36:18.494580 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494497    4925 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:36:18.494613 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494600    4925 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 18:36:18.494641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494610    4925 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 18:36:18.494693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494684    4925 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 18:36:18.494723 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494694    4925 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 18:36:18.494723 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494699    4925 policy_none.go:50] "Start"
Oct 30 18:36:18.494723 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494707    4925 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:36:18.494723 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494716    4925 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:36:18.494838 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494829    4925 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 18:36:18.494867 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494838    4925 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 18:36:18.494867 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.494843    4925 policy_none.go:44] "Start"
Oct 30 18:36:18.498994 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.498969    4925 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:36:18.502109 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.502079    4925 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:36:18.502203 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:18.502127    4925 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:36:18.502203 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.502139    4925 server.go:87] "Starting device plugin registration server"
Oct 30 18:36:18.503141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.503127    4925 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:36:18.503190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.503144    4925 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:36:18.503249 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.503231    4925 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:36:18.503361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.503327    4925 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:36:18.503428 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.503372    4925 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:36:18.503428 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.503379    4925 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:36:18.503515 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.503432    4925 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:36:18.503360202Z","UUID":"ccc74cae-0ddc-4273-aae0-fafa53dda4d3","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:36:18.504715 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.504697    4925 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:36:18.504792 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.504724    4925 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:36:18.504938 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:18.504919    4925 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:36:18.514886 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.514852    4925 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"]
Oct 30 18:36:18.517911 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.517892    4925 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.518888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.518847    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal" event={"ID":"f3cd2cf92111a4cf6f6c288eae5fe1a5","Type":"ContainerStarted","Data":"eab93ead3eab96d49f3683028c7ed7c1151005863fa6a277a03cb1b364df2000"}
Oct 30 18:36:18.518981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.518902    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal" event={"ID":"f3cd2cf92111a4cf6f6c288eae5fe1a5","Type":"ContainerDied","Data":"c5693126742fbfe3c07fa4c84b9b521b2c4215309736ab1c2fc8bb7010202e48"}
Oct 30 18:36:18.518981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.518922    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal" event={"ID":"f3cd2cf92111a4cf6f6c288eae5fe1a5","Type":"ContainerStarted","Data":"faa1dd80e2e92bf9d05e3aea03b5bd5733995ad7888281b0af8b4079958531c0"}
Oct 30 18:36:18.518981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.518940    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal" event={"ID":"749244aef4ca28f482f48e6f6d30c5f8","Type":"ContainerStarted","Data":"e495d1b3dd75d2ebc46b76ff26b9f2b8e11f197c78870c255e2dc9f50f4865ef"}
Oct 30 18:36:18.518981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.518954    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal" event={"ID":"749244aef4ca28f482f48e6f6d30c5f8","Type":"ContainerStarted","Data":"8aed94ba983787ab53fa85bf2c419096c9b05b013a9bbd1746f293ec8216d2d0"}
Oct 30 18:36:18.519192 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.519087    4925 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.535354 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.535337    4925 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:18.535431 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:18.535375    4925 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.535778 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.535765    4925 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:18.535822 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:18.535800    4925 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.595607 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.595585    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3cd2cf92111a4cf6f6c288eae5fe1a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"f3cd2cf92111a4cf6f6c288eae5fe1a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.595690 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.595609    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3cd2cf92111a4cf6f6c288eae5fe1a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"f3cd2cf92111a4cf6f6c288eae5fe1a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.595690 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.595626    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/749244aef4ca28f482f48e6f6d30c5f8-config\") pod \"kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"749244aef4ca28f482f48e6f6d30c5f8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.603698 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.603679    4925 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:18.605566 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.605551    4925 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:18.605638 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.605577    4925 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:18.605638 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.605590    4925 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-58.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:36:18.605638 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.605631    4925 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.617018 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.617001    4925 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.617068 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.617044    4925 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.696440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.696416    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3cd2cf92111a4cf6f6c288eae5fe1a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"f3cd2cf92111a4cf6f6c288eae5fe1a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.696544 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.696448    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/749244aef4ca28f482f48e6f6d30c5f8-config\") pod \"kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"749244aef4ca28f482f48e6f6d30c5f8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.696544 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.696472    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3cd2cf92111a4cf6f6c288eae5fe1a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"f3cd2cf92111a4cf6f6c288eae5fe1a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.696544 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.696524    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/749244aef4ca28f482f48e6f6d30c5f8-config\") pod \"kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"749244aef4ca28f482f48e6f6d30c5f8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.696655 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.696553    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3cd2cf92111a4cf6f6c288eae5fe1a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"f3cd2cf92111a4cf6f6c288eae5fe1a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:18.696655 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:18.696523    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f3cd2cf92111a4cf6f6c288eae5fe1a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" (UID: \"f3cd2cf92111a4cf6f6c288eae5fe1a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:19.366833 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.366801    4925 apiserver.go:52] "Watching apiserver"
Oct 30 18:36:19.377531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.377508    4925 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:36:19.378254 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.378227    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-qhk9m","openshift-console-operator/console-operator-7cd56b8dbc-68flm","openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg","openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf","kube-system/kas-connection-checker-685895c7c9-v788b","kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal","openshift-dns/dns-default-jcn7p","openshift-multus/multus-additional-cni-plugins-szvjp","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj","openshift-ingress-canary/ingress-canary-z2nsf","kube-system/kas-connection-checker-685895c7c9-985h5","openshift-multus/network-metrics-daemon-kjxws","openshift-network-operator/iptables-alerter-62g82","openshift-network-console/networking-console-plugin-86c478bb88-k9dgt","kube-system/kas-connection-checker-685895c7c9-tfwdw","openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8","openshift-multus/multus-jrtwp","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz","openshift-ingress/router-default-59c4cc76f-fbdqx","openshift-image-registry/node-ca-5kvjr","openshift-network-diagnostics/network-check-target-k6kkr","kube-system/global-pull-secret-syncer-cmf2j","openshift-cluster-node-tuning-operator/tuned-ktxbz","kube-system/konnectivity-agent-9pnlw","openshift-image-registry/image-registry-6684c84797-p6964","openshift-image-registry/image-registry-76f45597c6-hfv8h","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal","openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6","openshift-ovn-kubernetes/ovnkube-node-6hwlk","openshift-dns/node-resolver-6lkjf","openshift-insights/insights-runtime-extractor-9czjj"]
Oct 30 18:36:19.380158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.380133    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.380838 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.380819    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:19.384322 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.384296    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:19.384416 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.384328    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.384543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.384521    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.385068 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.385048    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 18:36:19.385239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.385188    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 18:36:19.385371 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.385328    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 18:36:19.385439 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.385399    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 18:36:19.385439 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.385405    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:19.385542 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.385480    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 18:36:19.385613 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.385594    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.385777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.385752    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 18:36:19.385919 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.385901    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.386000 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.385979    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.386082 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.386069    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 18:36:19.386177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.385983    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.386258 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.386243    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.386914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.386891    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 18:36:19.386981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.386942    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 18:36:19.387234 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.387213    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:36:19.387234 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.387230    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.387361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.387293    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.387361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.387309    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.387361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.387315    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.387361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.387312    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.387361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.387360    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.387585 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.387371    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 18:36:19.387585 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.387436    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.387585 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.387561    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:19.389450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.389431    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:19.392614 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.392585    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.393054 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393034    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 18:36:19.393160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393133    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 18:36:19.393380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393357    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 18:36:19.393457 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393404    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.393457 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393370    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.393457 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393368    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.393457 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393424    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 18:36:19.393657 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393505    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 18:36:19.393657 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393582    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 18:36:19.393657 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393617    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 18:36:19.393657 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393625    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.393657 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393634    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 18:36:19.393866 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393667    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.393866 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393686    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.393866 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393685    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 18:36:19.393866 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393714    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.393866 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.393785    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.394070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.394043    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 18:36:19.394158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.394146    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 18:36:19.394436 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.394415    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-bxft2\""
Oct 30 18:36:19.394495 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.394446    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 18:36:19.394600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.394581    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.394600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.394593    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:19.394600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.394599    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.394772 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.394636    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 18:36:19.395136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.395116    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 18:36:19.395512 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.395474    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:36:19.395648 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.395525    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:36:19.395648 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.395536    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-s4hwp\""
Oct 30 18:36:19.395648 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.395597    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.395797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.395714    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:36:19.395916 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.395897    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.396032 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.396012    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.396316 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.396280    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-xr42l\""
Oct 30 18:36:19.396406 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.396251    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:36:19.396526 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.396498    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:36:19.396616 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.396535    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vgk4h\""
Oct 30 18:36:19.396852 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.396836    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-89nks\""
Oct 30 18:36:19.396989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.396973    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-phzbf\""
Oct 30 18:36:19.397069 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.397034    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.397160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.397142    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8j87p\""
Oct 30 18:36:19.397217 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.397182    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:36:19.397263 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.397244    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dnnzq\""
Oct 30 18:36:19.397357 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.396982    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.397429 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.397388    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gjz7h\""
Oct 30 18:36:19.397489 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.397458    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:36:19.397674 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.397651    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.397725 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.397710    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d6n4c\""
Oct 30 18:36:19.397795 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.397778    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.397854 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.397836    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:36:19.397908 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.397890    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:36:19.398212 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.398192    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:36:19.398212 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.398222    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8njxf\""
Oct 30 18:36:19.398448 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.398193    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.398448 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.398335    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:36:19.398448 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.398198    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.398448 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.398396    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:36:19.398630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.398491    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:36:19.398630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.398546    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:36:19.398630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.398552    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:36:19.398630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.398593    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lfvmh\""
Oct 30 18:36:19.398826 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.398637    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.398826 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.398818    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ldjxj\""
Oct 30 18:36:19.399242 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.399203    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qs4k8\""
Oct 30 18:36:19.399316 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.399241    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sfbzk\""
Oct 30 18:36:19.399373 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.399357    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:36:19.399991 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.399967    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0ba138d-0a24-4f77-b1f4-c0861a48797f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:19.400025 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400006    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-default-certificate\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:19.400062 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400037    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:19.400132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400067    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5cf855c4-4e7e-4669-9adc-583e22e27e73-serving-cert\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:19.400184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400165    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b90705d-6abb-481d-8af0-89e542709329-serving-cert\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:19.400215 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400202    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4bee5094-7886-4fa9-aa7c-0ddc3d518619-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:19.400249 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400234    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-image-registry-private-configuration\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.400288 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400267    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-installation-pull-secrets\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.400380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400358    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5cf855c4-4e7e-4669-9adc-583e22e27e73-serving-cert\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:19.400430 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400402    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-default-certificate\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:19.400486 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400439    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:19.400486 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400474    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0b90705d-6abb-481d-8af0-89e542709329-snapshots\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:19.400688 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400513    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmgk4\" (UniqueName: \"kubernetes.io/projected/ae2cac8d-509e-4e22-b35c-ab6d662fdf24-kube-api-access-nmgk4\") pod \"kas-connection-checker-685895c7c9-tfwdw\" (UID: \"ae2cac8d-509e-4e22-b35c-ab6d662fdf24\") " pod="kube-system/kas-connection-checker-685895c7c9-tfwdw"
Oct 30 18:36:19.400688 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400518    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b90705d-6abb-481d-8af0-89e542709329-serving-cert\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:19.400688 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400544    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-installation-pull-secrets\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.400688 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400564    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-image-registry-private-configuration\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.400688 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400566    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-stats-auth\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:19.400688 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400664    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90705d-6abb-481d-8af0-89e542709329-service-ca-bundle\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:19.400986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400706    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.400986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400747    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5vxf\" (UniqueName: \"kubernetes.io/projected/f0ba138d-0a24-4f77-b1f4-c0861a48797f-kube-api-access-t5vxf\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:19.400986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400784    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd89g\" (UniqueName: \"kubernetes.io/projected/b0939980-01bb-4ae3-9c95-5e190b719cbe-kube-api-access-jd89g\") pod \"kas-connection-checker-685895c7c9-985h5\" (UID: \"b0939980-01bb-4ae3-9c95-5e190b719cbe\") " pod="kube-system/kas-connection-checker-685895c7c9-985h5"
Oct 30 18:36:19.400986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400812    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5cf855c4-4e7e-4669-9adc-583e22e27e73-config\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:19.400986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400817    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-stats-auth\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:19.400986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400846    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b52b9e-4797-4a2e-953e-7da0f61bca87-serving-cert\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:19.400986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400875    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0ba138d-0a24-4f77-b1f4-c0861a48797f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:19.400986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.400891    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0b90705d-6abb-481d-8af0-89e542709329-snapshots\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:19.401374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401044    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-bound-sa-token\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.401374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401083    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9mdl\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-kube-api-access-p9mdl\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.401374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401132    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90705d-6abb-481d-8af0-89e542709329-service-ca-bundle\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:19.401374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401141    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9mtz\" (UniqueName: \"kubernetes.io/projected/2c77e172-a80d-4eda-986d-ec8c3c52e131-kube-api-access-f9mtz\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:19.401374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401187    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5cf855c4-4e7e-4669-9adc-583e22e27e73-config\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:19.401374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401203    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:19.401374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401244    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90705d-6abb-481d-8af0-89e542709329-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:19.401647 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401595    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:19.401647 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401623    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d2b52b9e-4797-4a2e-953e-7da0f61bca87-serving-cert\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:19.401756 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401737    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4bee5094-7886-4fa9-aa7c-0ddc3d518619-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:19.401826 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401755    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgfnf\" (UniqueName: \"kubernetes.io/projected/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-kube-api-access-pgfnf\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:19.401826 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401799    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cf855c4-4e7e-4669-9adc-583e22e27e73-trusted-ca\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:19.401993 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401958    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97wlp\" (UniqueName: \"kubernetes.io/projected/d2b52b9e-4797-4a2e-953e-7da0f61bca87-kube-api-access-97wlp\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:19.402066 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401997    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84gss\" (UniqueName: \"kubernetes.io/projected/4bee5094-7886-4fa9-aa7c-0ddc3d518619-kube-api-access-84gss\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:19.402066 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402023    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-trusted-ca\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.402066 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402051    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:19.402265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402078    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl4tg\" (UniqueName: \"kubernetes.io/projected/5cf855c4-4e7e-4669-9adc-583e22e27e73-kube-api-access-tl4tg\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:19.402265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402128    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dn5r\" (UniqueName: \"kubernetes.io/projected/0b90705d-6abb-481d-8af0-89e542709329-kube-api-access-7dn5r\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:19.402265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402154    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jms4w\" (UniqueName: \"kubernetes.io/projected/4cd65688-51e1-4b8c-8dc5-6e3771a74e66-kube-api-access-jms4w\") pod \"volume-data-source-validator-58c9f5f46d-ljxjz\" (UID: \"4cd65688-51e1-4b8c-8dc5-6e3771a74e66\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz"
Oct 30 18:36:19.402265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402183    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd26f706-5971-4c47-a63e-699e7962ef8e-ca-trust-extracted\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.402265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402201    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cf855c4-4e7e-4669-9adc-583e22e27e73-trusted-ca\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:19.402265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402229    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-certificates\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.402265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402240    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.402265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402257    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b90705d-6abb-481d-8af0-89e542709329-tmp\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:19.402751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402285    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxx6s\" (UniqueName: \"kubernetes.io/projected/f16c549e-41e6-4721-ab5f-91b37f05a97c-kube-api-access-vxx6s\") pod \"kas-connection-checker-685895c7c9-v788b\" (UID: \"f16c549e-41e6-4721-ab5f-91b37f05a97c\") " pod="kube-system/kas-connection-checker-685895c7c9-v788b"
Oct 30 18:36:19.402751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402311    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d2b52b9e-4797-4a2e-953e-7da0f61bca87-config\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:19.402751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402335    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4bee5094-7886-4fa9-aa7c-0ddc3d518619-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:19.402751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.401961    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:19.402751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402517    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4bee5094-7886-4fa9-aa7c-0ddc3d518619-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:19.402751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402228    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b90705d-6abb-481d-8af0-89e542709329-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:19.402751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402544    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-certificates\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.402751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402574    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b90705d-6abb-481d-8af0-89e542709329-tmp\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:19.402751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402631    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd26f706-5971-4c47-a63e-699e7962ef8e-ca-trust-extracted\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.402751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402696    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d2b52b9e-4797-4a2e-953e-7da0f61bca87-config\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:19.403147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402811    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-trusted-ca\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.403147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402822    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.403147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402829    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:36:19.403147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402881    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-5rxk7\""
Oct 30 18:36:19.403147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.402939    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.405060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.405045    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:19.405163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.405150    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 18:36:19.405370 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.405358    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zsxst\""
Oct 30 18:36:19.405595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.405579    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9fb5r\""
Oct 30 18:36:19.405632 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.405589    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:36:19.405803 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.405789    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:36:19.407060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.407044    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmgk4\" (UniqueName: \"kubernetes.io/projected/ae2cac8d-509e-4e22-b35c-ab6d662fdf24-kube-api-access-nmgk4\") pod \"kas-connection-checker-685895c7c9-tfwdw\" (UID: \"ae2cac8d-509e-4e22-b35c-ab6d662fdf24\") " pod="kube-system/kas-connection-checker-685895c7c9-tfwdw"
Oct 30 18:36:19.407781 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.407766    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9mtz\" (UniqueName: \"kubernetes.io/projected/2c77e172-a80d-4eda-986d-ec8c3c52e131-kube-api-access-f9mtz\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:19.408028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.408008    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-bound-sa-token\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.408203 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.408188    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5vxf\" (UniqueName: \"kubernetes.io/projected/f0ba138d-0a24-4f77-b1f4-c0861a48797f-kube-api-access-t5vxf\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:19.408638 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.408623    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97wlp\" (UniqueName: \"kubernetes.io/projected/d2b52b9e-4797-4a2e-953e-7da0f61bca87-kube-api-access-97wlp\") pod \"service-ca-operator-544d749c8c-lhwcg\" (UID: \"d2b52b9e-4797-4a2e-953e-7da0f61bca87\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg"
Oct 30 18:36:19.408743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.408728    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgfnf\" (UniqueName: \"kubernetes.io/projected/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-kube-api-access-pgfnf\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:19.408817 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.408799    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9mdl\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-kube-api-access-p9mdl\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.409401 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.409378    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd89g\" (UniqueName: \"kubernetes.io/projected/b0939980-01bb-4ae3-9c95-5e190b719cbe-kube-api-access-jd89g\") pod \"kas-connection-checker-685895c7c9-985h5\" (UID: \"b0939980-01bb-4ae3-9c95-5e190b719cbe\") " pod="kube-system/kas-connection-checker-685895c7c9-985h5"
Oct 30 18:36:19.410806 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.410785    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxx6s\" (UniqueName: \"kubernetes.io/projected/f16c549e-41e6-4721-ab5f-91b37f05a97c-kube-api-access-vxx6s\") pod \"kas-connection-checker-685895c7c9-v788b\" (UID: \"f16c549e-41e6-4721-ab5f-91b37f05a97c\") " pod="kube-system/kas-connection-checker-685895c7c9-v788b"
Oct 30 18:36:19.410909 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.410888    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84gss\" (UniqueName: \"kubernetes.io/projected/4bee5094-7886-4fa9-aa7c-0ddc3d518619-kube-api-access-84gss\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc\" (UID: \"4bee5094-7886-4fa9-aa7c-0ddc3d518619\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc"
Oct 30 18:36:19.410946 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.410931    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl4tg\" (UniqueName: \"kubernetes.io/projected/5cf855c4-4e7e-4669-9adc-583e22e27e73-kube-api-access-tl4tg\") pod \"console-operator-7cd56b8dbc-68flm\" (UID: \"5cf855c4-4e7e-4669-9adc-583e22e27e73\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:19.411173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.411154    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jms4w\" (UniqueName: \"kubernetes.io/projected/4cd65688-51e1-4b8c-8dc5-6e3771a74e66-kube-api-access-jms4w\") pod \"volume-data-source-validator-58c9f5f46d-ljxjz\" (UID: \"4cd65688-51e1-4b8c-8dc5-6e3771a74e66\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-ljxjz"
Oct 30 18:36:19.411209 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.411157    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dn5r\" (UniqueName: \"kubernetes.io/projected/0b90705d-6abb-481d-8af0-89e542709329-kube-api-access-7dn5r\") pod \"insights-operator-7c4986d449-qhk9m\" (UID: \"0b90705d-6abb-481d-8af0-89e542709329\") " pod="openshift-insights/insights-operator-7c4986d449-qhk9m"
Oct 30 18:36:19.468007 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.467988    4925 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:19.468119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.468109    4925 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:19.480307 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.480260    4925 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:19.480451 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.480307    4925 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:19.480510 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.480457    4925 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:19.480510 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.480488    4925 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-58.us-east-2.compute.internal"
Oct 30 18:36:19.485262 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.485245    4925 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:36:19.502531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.502482    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9ndr\" (UniqueName: \"kubernetes.io/projected/74a6aabd-b72d-422d-ac40-7d07a9e56a41-kube-api-access-z9ndr\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:19.502531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.502516    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-system-cni-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.502729 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.502588    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-conf-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.502729 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.502642    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-multus-certs\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.502729 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.502667    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysconfig\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.502729 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.502693    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqjpj\" (UniqueName: \"kubernetes.io/projected/edd196c7-d883-4462-b422-385570ed9169-kube-api-access-tqjpj\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.502915 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.502747    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-crio-socket\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.502915 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.502853    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.502915 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.502883    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-systemd\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.503069 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.502916    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-env-overrides\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.503069 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.502960    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-socket-dir-parent\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.503069 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503020    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:19.503069 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503050    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgqmm\" (UniqueName: \"kubernetes.io/projected/3e7518c9-9163-41ac-9546-556b1773cec6-kube-api-access-vgqmm\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.503241 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.503131    4925 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:36:19.503241 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503160    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-env-overrides\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.503241 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503170    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnnxf\" (UniqueName: \"kubernetes.io/projected/9e5ac899-ecba-4c42-aa7a-e34e14269efa-kube-api-access-lnnxf\") pod \"network-check-source-6f78d59f99-hjr2n\" (UID: \"9e5ac899-ecba-4c42-aa7a-e34e14269efa\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n"
Oct 30 18:36:19.503241 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503228    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-host\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:36:19.503383 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.503268    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:20.003247335 +0000 UTC m=+1.719592390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : secret "router-metrics-certs-default" not found
Oct 30 18:36:19.503383 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503338    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-iptables-alerter-script\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:36:19.503383 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503372    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-lib-modules\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.503529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503403    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-slash\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.503529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503463    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7563c219-088f-4c66-81f5-86c581f08f34-hosts-file\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:36:19.503529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503487    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysctl-conf\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.503529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503522    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-trusted-ca\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.503753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503576    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734db718-c901-4fe7-a8e1-b90823ccd66c-tmp-dir\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:19.503753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503608    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-registration-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.503753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503648    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-iptables-alerter-script\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:36:19.503753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503670    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.503753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503677    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/734db718-c901-4fe7-a8e1-b90823ccd66c-tmp-dir\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:19.503753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503715    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-cni-netd\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.504038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503760    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.504038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503793    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-sys-fs\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.504038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503808    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-cni-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.504038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503830    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-kubernetes\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.504038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503868    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edd196c7-d883-4462-b422-385570ed9169-tmp\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.504038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503869    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-trusted-ca\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.504038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503910    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9q7l\" (UniqueName: \"kubernetes.io/projected/734db718-c901-4fe7-a8e1-b90823ccd66c-kube-api-access-s9q7l\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:19.504038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503952    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-etc-selinux\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.504038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503970    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/edd196c7-d883-4462-b422-385570ed9169-tmp\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.504038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.503973    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-systemd-units\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.504038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504037    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-run-netns\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504078    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-ovn\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504170    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504206    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-image-registry-private-configuration\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504235    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-registry-certificates\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504266    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504310    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpvfc\" (UniqueName: \"kubernetes.io/projected/4c1460ce-789a-411c-b748-2b3506811aa1-kube-api-access-kpvfc\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504334    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tc4s\" (UniqueName: \"kubernetes.io/projected/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-kube-api-access-7tc4s\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504362    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-image-registry-private-configuration\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504367    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-cnibin\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504379    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-registry-certificates\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.504401    4925 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504414    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-data-volume\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504432    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-node-log\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504448    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-system-cni-dir\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.504460    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls podName:f0ba138d-0a24-4f77-b1f4-c0861a48797f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:20.004444521 +0000 UTC m=+1.720789576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n92pf" (UID: "f0ba138d-0a24-4f77-b1f4-c0861a48797f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504516    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-kubelet\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504525    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-data-volume\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504545    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/36d759c0-ed6e-4170-aa8f-70560074a467-multus-daemon-config\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.504595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504582    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-systemd\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504616    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-installation-pull-secrets\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504640    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/734db718-c901-4fe7-a8e1-b90823ccd66c-config-volume\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504669    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-etc-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504693    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504718    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-574hk\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-kube-api-access-574hk\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504743    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504767    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-cni-bin\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504788    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/272efc3d-7980-4fd1-a20d-9bd023f48bec-cert\") pod \"ingress-canary-z2nsf\" (UID: \"272efc3d-7980-4fd1-a20d-9bd023f48bec\") " pod="openshift-ingress-canary/ingress-canary-z2nsf"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504825    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/734db718-c901-4fe7-a8e1-b90823ccd66c-config-volume\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504837    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/36d759c0-ed6e-4170-aa8f-70560074a467-multus-daemon-config\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504787    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-installation-pull-secrets\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504811    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-modprobe-d\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504895    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-log-socket\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.504911    4925 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.504922    4925 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f45597c6-hfv8h: secret "image-registry-tls" not found
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504948    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-ovnkube-script-lib\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504969    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/272efc3d-7980-4fd1-a20d-9bd023f48bec-cert\") pod \"ingress-canary-z2nsf\" (UID: \"272efc3d-7980-4fd1-a20d-9bd023f48bec\") " pod="openshift-ingress-canary/ingress-canary-z2nsf"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504970    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79swd\" (UniqueName: \"kubernetes.io/projected/70da42f1-d38f-490e-8db0-4c1d141b2009-kube-api-access-79swd\") pod \"migrator-66df5d496b-4lrn8\" (UID: \"70da42f1-d38f-490e-8db0-4c1d141b2009\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.504997    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-hostroot\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505011    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-run\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.505065    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls podName:cd26f706-5971-4c47-a63e-699e7962ef8e nodeName:}" failed. No retries permitted until 2026-10-30 18:36:20.005050474 +0000 UTC m=+1.721395572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls") pod "image-registry-76f45597c6-hfv8h" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e") : secret "image-registry-tls" not found
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505129    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505156    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e7518c9-9163-41ac-9546-556b1773cec6-ovn-node-metrics-cert\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505162    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-ovnkube-script-lib\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505194    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505291    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e7518c9-9163-41ac-9546-556b1773cec6-ovn-node-metrics-cert\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505312    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505346    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-device-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505372    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-var-lib-kubelet\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505419    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:19.505440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505442    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505469    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-cni-binary-copy\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505484    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3762f795-944b-458e-9458-c26eaf133871-agent-certs\") pod \"konnectivity-agent-9pnlw\" (UID: \"3762f795-944b-458e-9458-c26eaf133871\") " pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.505500    4925 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505499    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac85734e-3f97-473f-b3ab-8aadd898f381-kubelet-config\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.505542    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls podName:9e9032e7-1ec1-4a5d-8e28-622d4fd01392 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:20.005532102 +0000 UTC m=+1.721877156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-vzrmj" (UID: "9e9032e7-1ec1-4a5d-8e28-622d4fd01392") : secret "samples-operator-tls" not found
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505565    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkr9h\" (UniqueName: \"kubernetes.io/projected/272efc3d-7980-4fd1-a20d-9bd023f48bec-kube-api-access-dkr9h\") pod \"ingress-canary-z2nsf\" (UID: \"272efc3d-7980-4fd1-a20d-9bd023f48bec\") " pod="openshift-ingress-canary/ingress-canary-z2nsf"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505581    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysctl-d\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505601    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/edd196c7-d883-4462-b422-385570ed9169-etc-tuned\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505633    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3762f795-944b-458e-9458-c26eaf133871-konnectivity-ca\") pod \"konnectivity-agent-9pnlw\" (UID: \"3762f795-944b-458e-9458-c26eaf133871\") " pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505662    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-k8s-cni-cncf-io\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505681    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/edd196c7-d883-4462-b422-385570ed9169-etc-tuned\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505715    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-cni-multus\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505830    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-cni-binary-copy\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505840    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3762f795-944b-458e-9458-c26eaf133871-agent-certs\") pod \"konnectivity-agent-9pnlw\" (UID: \"3762f795-944b-458e-9458-c26eaf133871\") " pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505857    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3762f795-944b-458e-9458-c26eaf133871-konnectivity-ca\") pod \"konnectivity-agent-9pnlw\" (UID: \"3762f795-944b-458e-9458-c26eaf133871\") " pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505828    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fn8p\" (UniqueName: \"kubernetes.io/projected/36d759c0-ed6e-4170-aa8f-70560074a467-kube-api-access-6fn8p\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505900    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505930    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7563c219-088f-4c66-81f5-86c581f08f34-tmp-dir\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505955    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-bound-sa-token\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.505980    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506007    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-socket-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506019    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7563c219-088f-4c66-81f5-86c581f08f34-tmp-dir\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506032    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.506074    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:20.006063672 +0000 UTC m=+1.722408726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : configmap references non-existent config key: service-ca.crt
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506146    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-netns\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506183    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-etc-kubernetes\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506225    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-serviceca\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506247    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztnjp\" (UniqueName: \"kubernetes.io/projected/7563c219-088f-4c66-81f5-86c581f08f34-kube-api-access-ztnjp\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506280    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-host-slash\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506299    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w86dj\" (UniqueName: \"kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj\") pod \"network-check-target-k6kkr\" (UID: \"a7be31dd-dd7b-448e-938b-ba7a0807bf49\") " pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506316    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7bxj\" (UniqueName: \"kubernetes.io/projected/628d4dcc-30b9-4817-815e-53d36818b7c2-kube-api-access-k7bxj\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506323    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac85734e-3f97-473f-b3ab-8aadd898f381-original-pull-secret\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506332    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-kubelet\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506396    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26kg9\" (UniqueName: \"kubernetes.io/projected/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-api-access-26kg9\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506404    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-serviceca\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506451    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506482    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z97l8\" (UniqueName: \"kubernetes.io/projected/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-kube-api-access-z97l8\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506519    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-var-lib-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506547    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-ovnkube-config\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506599    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506621    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-cnibin\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506652    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36d759c0-ed6e-4170-aa8f-70560074a467-cni-binary-copy\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506672    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506688    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f478aac2-de91-4b13-836b-ba09809222f0-nginx-conf\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506724    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-sys\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506752    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3e7518c9-9163-41ac-9546-556b1773cec6-ovnkube-config\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506800    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac85734e-3f97-473f-b3ab-8aadd898f381-dbus\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506830    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506853    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-host\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506878    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-cni-bin\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506902    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e32eb376-a16f-421c-8958-a410a5650964-ca-trust-extracted\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506830    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36d759c0-ed6e-4170-aa8f-70560074a467-cni-binary-copy\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.506888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506900    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f478aac2-de91-4b13-836b-ba09809222f0-nginx-conf\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:19.508529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.506964    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/628d4dcc-30b9-4817-815e-53d36818b7c2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.508529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.507036    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e32eb376-a16f-421c-8958-a410a5650964-ca-trust-extracted\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.513267 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.513242    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9ndr\" (UniqueName: \"kubernetes.io/projected/74a6aabd-b72d-422d-ac40-7d07a9e56a41-kube-api-access-z9ndr\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:19.513345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.513288    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgqmm\" (UniqueName: \"kubernetes.io/projected/3e7518c9-9163-41ac-9546-556b1773cec6-kube-api-access-vgqmm\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.513345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.513313    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqjpj\" (UniqueName: \"kubernetes.io/projected/edd196c7-d883-4462-b422-385570ed9169-kube-api-access-tqjpj\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.513438 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.513424    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnnxf\" (UniqueName: \"kubernetes.io/projected/9e5ac899-ecba-4c42-aa7a-e34e14269efa-kube-api-access-lnnxf\") pod \"network-check-source-6f78d59f99-hjr2n\" (UID: \"9e5ac899-ecba-4c42-aa7a-e34e14269efa\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-hjr2n"
Oct 30 18:36:19.514357 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.514337    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7bxj\" (UniqueName: \"kubernetes.io/projected/628d4dcc-30b9-4817-815e-53d36818b7c2-kube-api-access-k7bxj\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.514576 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.514559    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9q7l\" (UniqueName: \"kubernetes.io/projected/734db718-c901-4fe7-a8e1-b90823ccd66c-kube-api-access-s9q7l\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:19.515086 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.515067    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkr9h\" (UniqueName: \"kubernetes.io/projected/272efc3d-7980-4fd1-a20d-9bd023f48bec-kube-api-access-dkr9h\") pod \"ingress-canary-z2nsf\" (UID: \"272efc3d-7980-4fd1-a20d-9bd023f48bec\") " pod="openshift-ingress-canary/ingress-canary-z2nsf"
Oct 30 18:36:19.515402 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.515384    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79swd\" (UniqueName: \"kubernetes.io/projected/70da42f1-d38f-490e-8db0-4c1d141b2009-kube-api-access-79swd\") pod \"migrator-66df5d496b-4lrn8\" (UID: \"70da42f1-d38f-490e-8db0-4c1d141b2009\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lrn8"
Oct 30 18:36:19.515923 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.515834    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpvfc\" (UniqueName: \"kubernetes.io/projected/4c1460ce-789a-411c-b748-2b3506811aa1-kube-api-access-kpvfc\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.516026 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.515961    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tc4s\" (UniqueName: \"kubernetes.io/projected/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-kube-api-access-7tc4s\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:36:19.516125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.516025    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z97l8\" (UniqueName: \"kubernetes.io/projected/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-kube-api-access-z97l8\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:36:19.516500 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.516440    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztnjp\" (UniqueName: \"kubernetes.io/projected/7563c219-088f-4c66-81f5-86c581f08f34-kube-api-access-ztnjp\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:36:19.516609 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.516495    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fn8p\" (UniqueName: \"kubernetes.io/projected/36d759c0-ed6e-4170-aa8f-70560074a467-kube-api-access-6fn8p\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.516609 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.516519    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26kg9\" (UniqueName: \"kubernetes.io/projected/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-api-access-26kg9\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.516801 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.516668    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-574hk\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-kube-api-access-574hk\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.516986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.516885    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-bound-sa-token\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.517880 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.517788    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w86dj\" (UniqueName: \"kubernetes.io/projected/a7be31dd-dd7b-448e-938b-ba7a0807bf49-kube-api-access-w86dj\") pod \"network-check-target-k6kkr\" (UID: \"a7be31dd-dd7b-448e-938b-ba7a0807bf49\") " pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:19.607313 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607293    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-registration-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.607398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607320    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.607398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607334    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-cni-netd\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.607398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607350    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.607398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607364    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-sys-fs\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.607398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607379    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-cni-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607400    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-kubernetes\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607428    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-etc-selinux\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607441    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607452    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-systemd-units\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607442    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-cni-netd\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607426    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-registration-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607469    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-cni-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607482    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-etc-selinux\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607395    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-run-ovn-kubernetes\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607485    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-run-netns\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607511    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-run-netns\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607518    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-systemd-units\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607448    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-sys-fs\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607518    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-kubernetes\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607565    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-ovn\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607597    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607605    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-ovn\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607640    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-cnibin\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607655    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-node-log\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.607649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607654    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607679    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-node-log\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607680    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-cnibin\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607692    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-system-cni-dir\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607718    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-kubelet\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607741    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-systemd\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607764    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-system-cni-dir\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607765    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-etc-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607774    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-kubelet\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607781    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-systemd\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607786    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607797    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-etc-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607810    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-cni-bin\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607826    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-modprobe-d\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607840    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-log-socket\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607854    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-hostroot\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607867    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-run\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607868    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-cni-bin\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607878    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-log-socket\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607886    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.607936    4925 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.607949    4925 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6684c84797-p6964: secret "image-registry-tls" not found
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607950    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-modprobe-d\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607957    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-run-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607965    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607986    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-run\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.607985    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-hostroot\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.608000    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls podName:e32eb376-a16f-421c-8958-a410a5650964 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:20.107984966 +0000 UTC m=+1.824330038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls") pod "image-registry-6684c84797-p6964" (UID: "e32eb376-a16f-421c-8958-a410a5650964") : secret "image-registry-tls" not found
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608019    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.608052    4925 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608082    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-device-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608133    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-device-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.608156    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls podName:e7b7aaa9-68a9-4259-a9ce-001217dfa67f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:20.108144586 +0000 UTC m=+1.824489640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9czjj" (UID: "e7b7aaa9-68a9-4259-a9ce-001217dfa67f") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.608182    4925 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608194    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-var-lib-kubelet\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608232    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-var-lib-kubelet\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.608268    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls podName:734db718-c901-4fe7-a8e1-b90823ccd66c nodeName:}" failed. No retries permitted until 2026-10-30 18:36:20.108259774 +0000 UTC m=+1.824604845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls") pod "dns-default-jcn7p" (UID: "734db718-c901-4fe7-a8e1-b90823ccd66c") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608296    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608326    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac85734e-3f97-473f-b3ab-8aadd898f381-kubelet-config\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608358    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysctl-d\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608369    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac85734e-3f97-473f-b3ab-8aadd898f381-kubelet-config\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608383    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-k8s-cni-cncf-io\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608407    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-cni-multus\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608435    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608459    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-var-lib-cni-multus\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608473    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-socket-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.608450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608479    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysctl-d\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608439    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-k8s-cni-cncf-io\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608496    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-netns\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.608521    4925 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608535    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-netns\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608536    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c1460ce-789a-411c-b748-2b3506811aa1-socket-dir\") pod \"aws-ebs-csi-driver-node-g4hc6\" (UID: \"4c1460ce-789a-411c-b748-2b3506811aa1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g4hc6"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608519    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-etc-kubernetes\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608559    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-etc-kubernetes\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.608570    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert podName:f478aac2-de91-4b13-836b-ba09809222f0 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:20.108557684 +0000 UTC m=+1.824902737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k9dgt" (UID: "f478aac2-de91-4b13-836b-ba09809222f0") : secret "networking-console-plugin-cert" not found
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608602    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-host-slash\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608621    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-kubelet\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608644    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-var-lib-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608656    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/800f854c-c40c-45d9-9ffa-b7ca3ac38d51-host-slash\") pod \"iptables-alerter-62g82\" (UID: \"800f854c-c40c-45d9-9ffa-b7ca3ac38d51\") " pod="openshift-network-operator/iptables-alerter-62g82"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608682    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-kubelet\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608659    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608710    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-var-lib-openvswitch\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608713    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-cnibin\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.608730    4925 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608741    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-cnibin\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608752    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-sys\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:19.608782    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs podName:74a6aabd-b72d-422d-ac40-7d07a9e56a41 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:20.108767434 +0000 UTC m=+1.825112495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs") pod "network-metrics-daemon-kjxws" (UID: "74a6aabd-b72d-422d-ac40-7d07a9e56a41") : secret "metrics-daemon-secret" not found
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608798    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-sys\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608801    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac85734e-3f97-473f-b3ab-8aadd898f381-dbus\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608805    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608830    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-host\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608848    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-cni-bin\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608856    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac85734e-3f97-473f-b3ab-8aadd898f381-dbus\") pod \"global-pull-secret-syncer-cmf2j\" (UID: \"ac85734e-3f97-473f-b3ab-8aadd898f381\") " pod="kube-system/global-pull-secret-syncer-cmf2j"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608864    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-system-cni-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608878    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-conf-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608907    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-multus-certs\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608920    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-conf-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608932    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-system-cni-dir\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608934    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysconfig\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608942    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-cni-bin\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608948    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-host\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608963    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-host-run-multus-certs\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608968    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-crio-socket\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608967    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysconfig\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.608994    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609012    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-systemd\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609026    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-socket-dir-parent\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609049    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-host\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609027    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-crio-socket\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609052    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-systemd\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609055    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/628d4dcc-30b9-4817-815e-53d36818b7c2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-szvjp\" (UID: \"628d4dcc-30b9-4817-815e-53d36818b7c2\") " pod="openshift-multus/multus-additional-cni-plugins-szvjp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609050    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/36d759c0-ed6e-4170-aa8f-70560074a467-multus-socket-dir-parent\") pod \"multus-jrtwp\" (UID: \"36d759c0-ed6e-4170-aa8f-70560074a467\") " pod="openshift-multus/multus-jrtwp"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609063    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-lib-modules\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609122    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3-host\") pod \"node-ca-5kvjr\" (UID: \"a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3\") " pod="openshift-image-registry/node-ca-5kvjr"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609139    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-slash\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609157    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-lib-modules\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609168    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7563c219-088f-4c66-81f5-86c581f08f34-hosts-file\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609193    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysctl-conf\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609163    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e7518c9-9163-41ac-9546-556b1773cec6-host-slash\") pod \"ovnkube-node-6hwlk\" (UID: \"3e7518c9-9163-41ac-9546-556b1773cec6\") " pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609232    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7563c219-088f-4c66-81f5-86c581f08f34-hosts-file\") pod \"node-resolver-6lkjf\" (UID: \"7563c219-088f-4c66-81f5-86c581f08f34\") " pod="openshift-dns/node-resolver-6lkjf"
Oct 30 18:36:19.610035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.609360    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/edd196c7-d883-4462-b422-385570ed9169-etc-sysctl-conf\") pod \"tuned-ktxbz\" (UID: \"edd196c7-d883-4462-b422-385570ed9169\") " pod="openshift-cluster-node-tuning-operator/tuned-ktxbz"
Oct 30 18:36:19.679797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.679781    4925 scope.go:122] "RemoveContainer" containerID="70c2560feead44db5071af9e4bce3fdbad88ffcafac5d9145f84a398128d94bb"
Oct 30 18:36:19.795283 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.795253    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:19.969231 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:19.969172    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:20.012166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.012136    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:20.012261 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.012194    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:20.012330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.012256    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:20.012330 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.012274    4925 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:20.012330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.012306    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:20.012330 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.012329    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls podName:f0ba138d-0a24-4f77-b1f4-c0861a48797f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:21.012315742 +0000 UTC m=+2.728660810 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n92pf" (UID: "f0ba138d-0a24-4f77-b1f4-c0861a48797f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:20.012517 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.012339    4925 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:20.012517 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.012352    4925 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76f45597c6-hfv8h: secret "image-registry-tls" not found
Oct 30 18:36:20.012517 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.012387    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls podName:cd26f706-5971-4c47-a63e-699e7962ef8e nodeName:}" failed. No retries permitted until 2026-10-30 18:36:21.012376545 +0000 UTC m=+2.728721598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls") pod "image-registry-76f45597c6-hfv8h" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e") : secret "image-registry-tls" not found
Oct 30 18:36:20.012517 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.012411    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:21.012396557 +0000 UTC m=+2.728741614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : configmap references non-existent config key: service-ca.crt
Oct 30 18:36:20.012517 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.012427    4925 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:36:20.012517 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.012458    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls podName:9e9032e7-1ec1-4a5d-8e28-622d4fd01392 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:21.012448541 +0000 UTC m=+2.728793595 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-vzrmj" (UID: "9e9032e7-1ec1-4a5d-8e28-622d4fd01392") : secret "samples-operator-tls" not found
Oct 30 18:36:20.012517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.012456    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:20.012517 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.012497    4925 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:36:20.012517 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.012524    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:21.012517875 +0000 UTC m=+2.728862929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : secret "router-metrics-certs-default" not found
Oct 30 18:36:20.112976 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.112951    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:20.113074 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.113029    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:20.113074 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.113058    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:20.113228 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.113074    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:20.113228 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.113118    4925 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:36:20.113228 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.113130    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:20.113228 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.113170    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs podName:74a6aabd-b72d-422d-ac40-7d07a9e56a41 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:21.113156039 +0000 UTC m=+2.829501097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs") pod "network-metrics-daemon-kjxws" (UID: "74a6aabd-b72d-422d-ac40-7d07a9e56a41") : secret "metrics-daemon-secret" not found
Oct 30 18:36:20.113228 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.113181    4925 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:20.113228 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.113205    4925 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:36:20.113228 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.113185    4925 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:36:20.113228 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.113231    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls podName:e7b7aaa9-68a9-4259-a9ce-001217dfa67f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:21.113217919 +0000 UTC m=+2.829562972 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9czjj" (UID: "e7b7aaa9-68a9-4259-a9ce-001217dfa67f") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:20.113516 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.113238    4925 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6684c84797-p6964: secret "image-registry-tls" not found
Oct 30 18:36:20.113516 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.113252    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert podName:f478aac2-de91-4b13-836b-ba09809222f0 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:21.113237513 +0000 UTC m=+2.829582568 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k9dgt" (UID: "f478aac2-de91-4b13-836b-ba09809222f0") : secret "networking-console-plugin-cert" not found
Oct 30 18:36:20.113516 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.113255    4925 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:36:20.113516 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.113291    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls podName:e32eb376-a16f-421c-8958-a410a5650964 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:21.113278403 +0000 UTC m=+2.829623471 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls") pod "image-registry-6684c84797-p6964" (UID: "e32eb376-a16f-421c-8958-a410a5650964") : secret "image-registry-tls" not found
Oct 30 18:36:20.113516 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.113314    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls podName:734db718-c901-4fe7-a8e1-b90823ccd66c nodeName:}" failed. No retries permitted until 2026-10-30 18:36:21.113298143 +0000 UTC m=+2.829643198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls") pod "dns-default-jcn7p" (UID: "734db718-c901-4fe7-a8e1-b90823ccd66c") : secret "dns-default-metrics-tls" not found
Oct 30 18:36:20.475086 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.475061    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 18:36:20.475565 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.475549    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/1.log"
Oct 30 18:36:20.475634 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.475581    4925 generic.go:356] "Generic (PLEG): container finished" podID="5cf855c4-4e7e-4669-9adc-583e22e27e73" containerID="367991f4b6561209f1009db2afc4fe1ca5b58961ed62177fecc7ec83533f4830" exitCode=255
Oct 30 18:36:20.475680 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.475655    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm" event={"ID":"5cf855c4-4e7e-4669-9adc-583e22e27e73","Type":"ContainerDied","Data":"367991f4b6561209f1009db2afc4fe1ca5b58961ed62177fecc7ec83533f4830"}
Oct 30 18:36:20.475753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.475711    4925 scope.go:122] "RemoveContainer" containerID="70c2560feead44db5071af9e4bce3fdbad88ffcafac5d9145f84a398128d94bb"
Oct 30 18:36:20.476247 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.476231    4925 scope.go:122] "RemoveContainer" containerID="367991f4b6561209f1009db2afc4fe1ca5b58961ed62177fecc7ec83533f4830"
Oct 30 18:36:20.476453 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:20.476417    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-68flm_openshift-console-operator(5cf855c4-4e7e-4669-9adc-583e22e27e73)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm" podUID="5cf855c4-4e7e-4669-9adc-583e22e27e73"
Oct 30 18:36:20.524366 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.524339    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:20.721917 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.721864    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:20.944189 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:20.944163    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:21.020174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.020146    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:21.020299 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.020204    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:21.020299 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.020257    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:21.020299 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:21.020281    4925 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:36:21.020442 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:21.020337    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:23.020323104 +0000 UTC m=+4.736668162 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : secret "router-metrics-certs-default" not found
Oct 30 18:36:21.020442 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:21.020358    4925 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:21.020442 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:21.020417    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls podName:f0ba138d-0a24-4f77-b1f4-c0861a48797f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:23.020402697 +0000 UTC m=+4.736747799 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-n92pf" (UID: "f0ba138d-0a24-4f77-b1f4-c0861a48797f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:36:21.020442 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.020438    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:21.020679 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.020490    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:21.020679 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:21.020598    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:23.02057432 +0000 UTC m=+4.736919414 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : configmap references non-existent config key: service-ca.crt
Oct 30 18:36:21.020679 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.020637    4925 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:36:21.023209 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.023187    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9e9032e7-1ec1-4a5d-8e28-622d4fd01392-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-vzrmj\" (UID: \"9e9032e7-1ec1-4a5d-8e28-622d4fd01392\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:21.023303 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.023260    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") pod \"image-registry-76f45597c6-hfv8h\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") " pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:21.121083 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.121053    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:21.121194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.121090    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:21.121194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.121130    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:21.121194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.121157    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:21.121194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.121191    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:21.121411 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:21.121261    4925 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:21.121411 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:21.121299    4925 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:36:21.121411 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:21.121323    4925 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:36:21.121411 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:21.121308    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls podName:e7b7aaa9-68a9-4259-a9ce-001217dfa67f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:23.121296835 +0000 UTC m=+4.837641889 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9czjj" (UID: "e7b7aaa9-68a9-4259-a9ce-001217dfa67f") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:21.121411 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:21.121369    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert podName:f478aac2-de91-4b13-836b-ba09809222f0 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:23.121354033 +0000 UTC m=+4.837699089 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k9dgt" (UID: "f478aac2-de91-4b13-836b-ba09809222f0") : secret "networking-console-plugin-cert" not found
Oct 30 18:36:21.121411 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:21.121386    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs podName:74a6aabd-b72d-422d-ac40-7d07a9e56a41 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:23.121375943 +0000 UTC m=+4.837720997 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs") pod "network-metrics-daemon-kjxws" (UID: "74a6aabd-b72d-422d-ac40-7d07a9e56a41") : secret "metrics-daemon-secret" not found
Oct 30 18:36:21.123566 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.123545    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") pod \"image-registry-6684c84797-p6964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") " pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:21.123834 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.123812    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/734db718-c901-4fe7-a8e1-b90823ccd66c-metrics-tls\") pod \"dns-default-jcn7p\" (UID: \"734db718-c901-4fe7-a8e1-b90823ccd66c\") " pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:21.142787 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.142762    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:21.180555 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.180531    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:21.187039 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.187022    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"
Oct 30 18:36:21.193495 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.193474    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:21.195350 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.195288    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:21.317860 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.317828    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:21.467465 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.467407    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76f45597c6-hfv8h"]
Oct 30 18:36:21.471983 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:21.471949    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd26f706_5971_4c47_a63e_699e7962ef8e.slice/crio-45edf1de7c18daa21ebe78f3fe1ca7dfffc903f9c30b86fb40ac347f8b72a6d5 WatchSource:0}: Error finding container 45edf1de7c18daa21ebe78f3fe1ca7dfffc903f9c30b86fb40ac347f8b72a6d5: Status 404 returned error can't find the container with id 45edf1de7c18daa21ebe78f3fe1ca7dfffc903f9c30b86fb40ac347f8b72a6d5
Oct 30 18:36:21.481043 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.481024    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 18:36:21.481380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.481342    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj"]
Oct 30 18:36:21.482458 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.482429    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f45597c6-hfv8h" event={"ID":"cd26f706-5971-4c47-a63e-699e7962ef8e","Type":"ContainerStarted","Data":"45edf1de7c18daa21ebe78f3fe1ca7dfffc903f9c30b86fb40ac347f8b72a6d5"}
Oct 30 18:36:21.489531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.489507    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6684c84797-p6964"]
Oct 30 18:36:21.492348 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.492326    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:21.496210 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.496191    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jcn7p"]
Oct 30 18:36:21.498074 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:21.498048    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode32eb376_a16f_421c_8958_a410a5650964.slice/crio-e6324a28990980d094a8a00dead461a059a559fadf98846fb1a8611e8bc20ef0 WatchSource:0}: Error finding container e6324a28990980d094a8a00dead461a059a559fadf98846fb1a8611e8bc20ef0: Status 404 returned error can't find the container with id e6324a28990980d094a8a00dead461a059a559fadf98846fb1a8611e8bc20ef0
Oct 30 18:36:21.499953 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:21.499917    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod734db718_c901_4fe7_a8e1_b90823ccd66c.slice/crio-da163d6036ae3301d0ef80559ac83948a550e76e2b748f703d4d23f6ccaffa36 WatchSource:0}: Error finding container da163d6036ae3301d0ef80559ac83948a550e76e2b748f703d4d23f6ccaffa36: Status 404 returned error can't find the container with id da163d6036ae3301d0ef80559ac83948a550e76e2b748f703d4d23f6ccaffa36
Oct 30 18:36:21.502526 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.502504    4925 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:36:21.974185 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:21.974151    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:22.012833 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.012804    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6lkjf_7563c219-088f-4c66-81f5-86c581f08f34/dns-node-resolver/0.log"
Oct 30 18:36:22.189688 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.189656    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:22.490659 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.490617    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f45597c6-hfv8h" event={"ID":"cd26f706-5971-4c47-a63e-699e7962ef8e","Type":"ContainerStarted","Data":"2026aceed699d170f6fb924c9495951809d1fba1c907a9a04102f6dd0dd7bd92"}
Oct 30 18:36:22.492937 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.492904    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jcn7p" event={"ID":"734db718-c901-4fe7-a8e1-b90823ccd66c","Type":"ContainerStarted","Data":"da163d6036ae3301d0ef80559ac83948a550e76e2b748f703d4d23f6ccaffa36"}
Oct 30 18:36:22.494949 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.494922    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6684c84797-p6964" event={"ID":"e32eb376-a16f-421c-8958-a410a5650964","Type":"ContainerStarted","Data":"a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e"}
Oct 30 18:36:22.495040 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.494967    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6684c84797-p6964" event={"ID":"e32eb376-a16f-421c-8958-a410a5650964","Type":"ContainerStarted","Data":"e6324a28990980d094a8a00dead461a059a559fadf98846fb1a8611e8bc20ef0"}
Oct 30 18:36:22.496444 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.496419    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj" event={"ID":"9e9032e7-1ec1-4a5d-8e28-622d4fd01392","Type":"ContainerStarted","Data":"4a7e068795f9fc7aca6f04ea8f9d1efd5a89f31ab77b9305a9ec7007864125e5"}
Oct 30 18:36:22.507980 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.507955    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:22.508291 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.508276    4925 scope.go:122] "RemoveContainer" containerID="367991f4b6561209f1009db2afc4fe1ca5b58961ed62177fecc7ec83533f4830"
Oct 30 18:36:22.508467 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:22.508448    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-68flm_openshift-console-operator(5cf855c4-4e7e-4669-9adc-583e22e27e73)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm" podUID="5cf855c4-4e7e-4669-9adc-583e22e27e73"
Oct 30 18:36:22.572676 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.572521    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:22.614048 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.614019    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6684c84797-p6964_e32eb376-a16f-421c-8958-a410a5650964/registry/0.log"
Oct 30 18:36:22.700757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.700729    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:22.776819 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.776679    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:22.813301 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.813271    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-76f45597c6-hfv8h_cd26f706-5971-4c47-a63e-699e7962ef8e/registry/0.log"
Oct 30 18:36:22.963626 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:22.963578    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:23.013552 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.013450    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5kvjr_a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3/node-ca/0.log"
Oct 30 18:36:23.041136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.041061    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:23.041268 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.041136    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:23.041268 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.041222    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:23.041406 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:23.041390    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:27.041371698 +0000 UTC m=+8.757716752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : configmap references non-existent config key: service-ca.crt
Oct 30 18:36:23.044281 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.044256    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c77e172-a80d-4eda-986d-ec8c3c52e131-metrics-certs\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:23.044388 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.044361    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0ba138d-0a24-4f77-b1f4-c0861a48797f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-n92pf\" (UID: \"f0ba138d-0a24-4f77-b1f4-c0861a48797f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:23.141926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.141899    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:23.142123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.141938    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:23.142123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.141971    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:23.142123 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:23.142023    4925 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:23.142123 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:23.142076    4925 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:36:23.142123 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:23.142110    4925 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:36:23.142381 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:23.142115    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls podName:e7b7aaa9-68a9-4259-a9ce-001217dfa67f nodeName:}" failed. No retries permitted until 2026-10-30 18:36:27.142075227 +0000 UTC m=+8.858420281 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9czjj" (UID: "e7b7aaa9-68a9-4259-a9ce-001217dfa67f") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:36:23.142381 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:23.142174    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs podName:74a6aabd-b72d-422d-ac40-7d07a9e56a41 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:27.1421614 +0000 UTC m=+8.858506455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs") pod "network-metrics-daemon-kjxws" (UID: "74a6aabd-b72d-422d-ac40-7d07a9e56a41") : secret "metrics-daemon-secret" not found
Oct 30 18:36:23.142381 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:23.142186    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert podName:f478aac2-de91-4b13-836b-ba09809222f0 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:27.14217969 +0000 UTC m=+8.858524743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k9dgt" (UID: "f478aac2-de91-4b13-836b-ba09809222f0") : secret "networking-console-plugin-cert" not found
Oct 30 18:36:23.228985 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.228963    4925 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:23.283414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.283315    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"
Oct 30 18:36:23.380782 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.380759    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:23.458192 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.458167    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:23.501137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.501111    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jcn7p" event={"ID":"734db718-c901-4fe7-a8e1-b90823ccd66c","Type":"ContainerStarted","Data":"14a4f5bb769780367d7b8e0ade7b8cf2edcbbea598fa7da837d0b5fe1014c2de"}
Oct 30 18:36:23.501531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.501169    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jcn7p" event={"ID":"734db718-c901-4fe7-a8e1-b90823ccd66c","Type":"ContainerStarted","Data":"63d934528607b8bb18956f57f6fad8562f0375af5483a92b862ad43194e64a0d"}
Oct 30 18:36:23.501531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.501203    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:23.502934 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.502917    4925 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:36:23.503026 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.502960    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj" event={"ID":"9e9032e7-1ec1-4a5d-8e28-622d4fd01392","Type":"ContainerStarted","Data":"b7a4c1077891331539f0588d6b46133ea6c38736e96e85e0c0d52d6122bdf590"}
Oct 30 18:36:23.503026 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.503010    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-vzrmj" event={"ID":"9e9032e7-1ec1-4a5d-8e28-622d4fd01392","Type":"ContainerStarted","Data":"7c9a88a259e83aa6ed29cd52fef75f9a764c05d9ceefcf60531f169d0016dd43"}
Oct 30 18:36:23.503176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.503072    4925 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:36:23.503517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.503502    4925 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:36:23.503517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.503511    4925 scope.go:122] "RemoveContainer" containerID="367991f4b6561209f1009db2afc4fe1ca5b58961ed62177fecc7ec83533f4830"
Oct 30 18:36:23.503649 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:23.503633    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-68flm_openshift-console-operator(5cf855c4-4e7e-4669-9adc-583e22e27e73)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm" podUID="5cf855c4-4e7e-4669-9adc-583e22e27e73"
Oct 30 18:36:23.537647 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.537625    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:23.558482 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.558434    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf"]
Oct 30 18:36:23.561838 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:23.561818    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ba138d_0a24_4f77_b1f4_c0861a48797f.slice/crio-1fa6e1c4e9085c4fcd9f00bd7c10dd27031d7e03bd5d5a274ae27a1667881444 WatchSource:0}: Error finding container 1fa6e1c4e9085c4fcd9f00bd7c10dd27031d7e03bd5d5a274ae27a1667881444: Status 404 returned error can't find the container with id 1fa6e1c4e9085c4fcd9f00bd7c10dd27031d7e03bd5d5a274ae27a1667881444
Oct 30 18:36:23.971141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:23.971115    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:24.016434 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:24.016415    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z2nsf_272efc3d-7980-4fd1-a20d-9bd023f48bec/serve-healthcheck-canary/0.log"
Oct 30 18:36:24.178524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:24.178481    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:24.214025 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:24.213979    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-4lrn8_70da42f1-d38f-490e-8db0-4c1d141b2009/migrator/0.log"
Oct 30 18:36:24.507176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:24.507141    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf" event={"ID":"f0ba138d-0a24-4f77-b1f4-c0861a48797f","Type":"ContainerStarted","Data":"1fa6e1c4e9085c4fcd9f00bd7c10dd27031d7e03bd5d5a274ae27a1667881444"}
Oct 30 18:36:24.570142 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:24.570085    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:24.617497 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:24.617467    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc_4bee5094-7886-4fa9-aa7c-0ddc3d518619/kube-storage-version-migrator-operator/0.log"
Oct 30 18:36:24.775718 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:24.775568    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:25.012045 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:25.012010    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:25.411860 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:25.411828    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:25.511909 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:25.511839    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-n92pf" event={"ID":"f0ba138d-0a24-4f77-b1f4-c0861a48797f","Type":"ContainerStarted","Data":"2dcd4ccb7484b36baea4f820b258aa26a931afd4adfb757b926d00293438bba7"}
Oct 30 18:36:25.686209 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:25.686180    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:25.867163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:25.867065    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:26.041263 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:26.041230    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:26.214720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:26.214676    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:26.573846 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:26.573775    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:26.772444 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:26.772389    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:26.862249 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:26.862193    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:27.077082 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.077054    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:27.077252 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:27.077234    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle podName:2c77e172-a80d-4eda-986d-ec8c3c52e131 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:35.077214916 +0000 UTC m=+16.793559987 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle") pod "router-default-59c4cc76f-fbdqx" (UID: "2c77e172-a80d-4eda-986d-ec8c3c52e131") : configmap references non-existent config key: service-ca.crt
Oct 30 18:36:27.178069 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.178032    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:27.178188 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.178087    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:27.178188 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:27.178177    4925 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:36:27.178264 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:27.178220    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert podName:f478aac2-de91-4b13-836b-ba09809222f0 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:35.17820817 +0000 UTC m=+16.894553237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k9dgt" (UID: "f478aac2-de91-4b13-836b-ba09809222f0") : secret "networking-console-plugin-cert" not found
Oct 30 18:36:27.178307 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.178284    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:27.180478 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.180457    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e7b7aaa9-68a9-4259-a9ce-001217dfa67f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9czjj\" (UID: \"e7b7aaa9-68a9-4259-a9ce-001217dfa67f\") " pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:27.180611 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.180586    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74a6aabd-b72d-422d-ac40-7d07a9e56a41-metrics-certs\") pod \"network-metrics-daemon-kjxws\" (UID: \"74a6aabd-b72d-422d-ac40-7d07a9e56a41\") " pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:27.189659 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.189639    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kjxws"
Oct 30 18:36:27.202810 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.202786    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9czjj"
Oct 30 18:36:27.466403 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.466330    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kjxws"]
Oct 30 18:36:27.469777 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:27.469752    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74a6aabd_b72d_422d_ac40_7d07a9e56a41.slice/crio-36f1f8160427c236ac044039a74fedf3890597f71d0d6740455bd826e1062921 WatchSource:0}: Error finding container 36f1f8160427c236ac044039a74fedf3890597f71d0d6740455bd826e1062921: Status 404 returned error can't find the container with id 36f1f8160427c236ac044039a74fedf3890597f71d0d6740455bd826e1062921
Oct 30 18:36:27.489562 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.489536    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9czjj"]
Oct 30 18:36:27.492325 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:27.492302    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b7aaa9_68a9_4259_a9ce_001217dfa67f.slice/crio-2ad6cac6d6b6ffe65c7ab46aaf30bfa559e0e5fb07160732a1aa46b13f39f43c WatchSource:0}: Error finding container 2ad6cac6d6b6ffe65c7ab46aaf30bfa559e0e5fb07160732a1aa46b13f39f43c: Status 404 returned error can't find the container with id 2ad6cac6d6b6ffe65c7ab46aaf30bfa559e0e5fb07160732a1aa46b13f39f43c
Oct 30 18:36:27.518878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.518850    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kjxws" event={"ID":"74a6aabd-b72d-422d-ac40-7d07a9e56a41","Type":"ContainerStarted","Data":"36f1f8160427c236ac044039a74fedf3890597f71d0d6740455bd826e1062921"}
Oct 30 18:36:27.520044 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.520012    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9czjj" event={"ID":"e7b7aaa9-68a9-4259-a9ce-001217dfa67f","Type":"ContainerStarted","Data":"2ad6cac6d6b6ffe65c7ab46aaf30bfa559e0e5fb07160732a1aa46b13f39f43c"}
Oct 30 18:36:27.686936 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.686895    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:27.967731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:27.967702    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:28.167954 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:28.167871    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:28.327155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:28.327126    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:28.524980 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:28.524957    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9czjj" event={"ID":"e7b7aaa9-68a9-4259-a9ce-001217dfa67f","Type":"ContainerStarted","Data":"f21ed7588b67259b0c3943bbd13ea6491413b92a30f327a2301c2c0d8cd24ea3"}
Oct 30 18:36:28.525079 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:28.525006    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9czjj" event={"ID":"e7b7aaa9-68a9-4259-a9ce-001217dfa67f","Type":"ContainerStarted","Data":"fb53b1d784e3e290bd947cb9ff4400721b5024eb8ba0bec5dfc318a913f6061f"}
Oct 30 18:36:28.576071 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:28.576040    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:28.769475 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:28.769391    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:29.376011 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:29.375972    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:29.530533 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:29.530505    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kjxws" event={"ID":"74a6aabd-b72d-422d-ac40-7d07a9e56a41","Type":"ContainerStarted","Data":"3b53dc989e292ab83f29072376c4b27f99fbeb3f116859596c219a16e72dad25"}
Oct 30 18:36:29.535082 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:29.530554    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kjxws" event={"ID":"74a6aabd-b72d-422d-ac40-7d07a9e56a41","Type":"ContainerStarted","Data":"09f5a93749699abef68f593f70472c1a4ac6c15536cf7f2ca7714b92b97c0e08"}
Oct 30 18:36:29.975380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:29.975343    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:30.211876 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:30.211843    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:30.414710 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:30.414692    4925 scope.go:122] "RemoveContainer" containerID="367991f4b6561209f1009db2afc4fe1ca5b58961ed62177fecc7ec83533f4830"
Oct 30 18:36:30.539106 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:30.539066    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9czjj" event={"ID":"e7b7aaa9-68a9-4259-a9ce-001217dfa67f","Type":"ContainerStarted","Data":"2b195733a429c299c4768565ce35743de41b53fa63c02dbc32b1b736f56f03b4"}
Oct 30 18:36:30.540914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:30.540895    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 18:36:30.541031 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:30.540995    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm" event={"ID":"5cf855c4-4e7e-4669-9adc-583e22e27e73","Type":"ContainerStarted","Data":"eb32d2e4cbe9899de7e0e0414d49ba569f26ed44a369659a8de6f6b544273636"}
Oct 30 18:36:30.541394 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:30.541372    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:30.578465 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:30.578446    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:30.762602 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:30.762548    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-68flm"
Oct 30 18:36:30.779695 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:30.779670    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:31.064304 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:31.064227    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:31.246803 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:31.246750    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:31.417108 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:31.417060    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:31.775799 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:31.775717    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:31.949982 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:31.949955    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:32.127998 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:32.127897    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:33.509325 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:33.509294    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jcn7p"
Oct 30 18:36:34.446698 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:34.446661    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76f45597c6-hfv8h"]
Oct 30 18:36:34.571901 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:34.571866    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:34.877003 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:34.876915    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:35.146371 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:35.146300    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:35.146838 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:35.146817    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c77e172-a80d-4eda-986d-ec8c3c52e131-service-ca-bundle\") pod \"router-default-59c4cc76f-fbdqx\" (UID: \"2c77e172-a80d-4eda-986d-ec8c3c52e131\") " pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:35.192626 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:35.192569    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:35.247057 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:35.247029    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:35.249622 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:35.249604    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f478aac2-de91-4b13-836b-ba09809222f0-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k9dgt\" (UID: \"f478aac2-de91-4b13-836b-ba09809222f0\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:35.285642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:35.285619    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:35.288182 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:35.288163    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"
Oct 30 18:36:35.390691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:35.390661    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:35.566763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:35.566735    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-k9dgt"]
Oct 30 18:36:35.570059 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:35.570026    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59c4cc76f-fbdqx"]
Oct 30 18:36:35.570059 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:35.570043    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf478aac2_de91_4b13_836b_ba09809222f0.slice/crio-7bdf832a097bab479e1c581cdba1ab33c8bdc381a486700c7438acbf4cad3f36 WatchSource:0}: Error finding container 7bdf832a097bab479e1c581cdba1ab33c8bdc381a486700c7438acbf4cad3f36: Status 404 returned error can't find the container with id 7bdf832a097bab479e1c581cdba1ab33c8bdc381a486700c7438acbf4cad3f36
Oct 30 18:36:35.573392 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:35.573345    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c77e172_a80d_4eda_986d_ec8c3c52e131.slice/crio-d0e167839c8938bde0a64571530548d8b3d7f5ae4c3064c3255f2eb6e2f6ebd0 WatchSource:0}: Error finding container d0e167839c8938bde0a64571530548d8b3d7f5ae4c3064c3255f2eb6e2f6ebd0: Status 404 returned error can't find the container with id d0e167839c8938bde0a64571530548d8b3d7f5ae4c3064c3255f2eb6e2f6ebd0
Oct 30 18:36:36.088066 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:36.088029    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:36.261259 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:36.261200    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:36.564039 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:36.564005    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt" event={"ID":"f478aac2-de91-4b13-836b-ba09809222f0","Type":"ContainerStarted","Data":"7bdf832a097bab479e1c581cdba1ab33c8bdc381a486700c7438acbf4cad3f36"}
Oct 30 18:36:36.565534 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:36.565510    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59c4cc76f-fbdqx" event={"ID":"2c77e172-a80d-4eda-986d-ec8c3c52e131","Type":"ContainerStarted","Data":"b87f6444b31159a094ef3f46ad363bbad5afe09bf1a7e3b5b004993616bb98a7"}
Oct 30 18:36:36.565664 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:36.565547    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59c4cc76f-fbdqx" event={"ID":"2c77e172-a80d-4eda-986d-ec8c3c52e131","Type":"ContainerStarted","Data":"d0e167839c8938bde0a64571530548d8b3d7f5ae4c3064c3255f2eb6e2f6ebd0"}
Oct 30 18:36:36.867287 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:36.867202    4925 patch_prober.go:29] interesting pod/image-registry-6684c84797-p6964 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 18:36:36.867287 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:36.867257    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6684c84797-p6964" podUID="e32eb376-a16f-421c-8958-a410a5650964" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:36:36.913519 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:36.913491    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:37.089027 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:37.088994    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:37.286291 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:37.286264    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:37.289191 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:37.289170    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:37.571449 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:37.571373    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-k9dgt" event={"ID":"f478aac2-de91-4b13-836b-ba09809222f0","Type":"ContainerStarted","Data":"dfdc16f58daf445350ee4a643fe6d1da62b917f2f9098762fdda9e2fc53e1814"}
Oct 30 18:36:37.571585 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:37.571462    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:37.572805 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:37.572779    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-59c4cc76f-fbdqx"
Oct 30 18:36:39.102360 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:39.102329    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6hwlk"
Oct 30 18:36:39.118208 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:39.118182    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9pnlw"
Oct 30 18:36:39.308163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:39.308134    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:39.522450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:39.522419    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:40.682479 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:40.682444    4925 patch_prober.go:29] interesting pod/image-registry-6684c84797-p6964 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 18:36:40.682902 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:40.682505    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6684c84797-p6964" podUID="e32eb376-a16f-421c-8958-a410a5650964" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:36:41.081458 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.081376    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:41.274997 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.274968    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bdfjd"]
Oct 30 18:36:41.280413 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.280391    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.320228 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.320198    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:41.323443 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.323425    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 18:36:41.323538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.323443    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 18:36:41.323600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.323587    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dgrqn\""
Oct 30 18:36:41.355194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.355144    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 18:36:41.355194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.355173    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 18:36:41.397566 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.397540    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kgvk\" (UniqueName: \"kubernetes.io/projected/1c47621d-ebe5-4469-a944-c74fd68df26f-kube-api-access-8kgvk\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.397646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.397576    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.397646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.397595    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-wtmp\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.397720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.397664    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-tls\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.397765 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.397750    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-textfile\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.397822 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.397810    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c47621d-ebe5-4469-a944-c74fd68df26f-sys\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.397855 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.397845    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c47621d-ebe5-4469-a944-c74fd68df26f-metrics-client-ca\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.397888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.397874    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c47621d-ebe5-4469-a944-c74fd68df26f-root\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.397924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.397890    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-accelerators-collector-config\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.498695 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.498671    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c47621d-ebe5-4469-a944-c74fd68df26f-metrics-client-ca\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.498789 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.498700    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c47621d-ebe5-4469-a944-c74fd68df26f-root\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.498789 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.498720    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-accelerators-collector-config\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.498789 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.498747    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kgvk\" (UniqueName: \"kubernetes.io/projected/1c47621d-ebe5-4469-a944-c74fd68df26f-kube-api-access-8kgvk\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.498789 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.498775    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.498947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.498790    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c47621d-ebe5-4469-a944-c74fd68df26f-root\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.498947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.498802    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-wtmp\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.498947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.498840    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-tls\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.498947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.498868    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-textfile\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.498947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.498921    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c47621d-ebe5-4469-a944-c74fd68df26f-sys\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.499225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.498990    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-wtmp\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.499225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.498996    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c47621d-ebe5-4469-a944-c74fd68df26f-sys\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.499339 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.499290    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-textfile\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.499339 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.499314    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-accelerators-collector-config\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.499339 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.499327    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c47621d-ebe5-4469-a944-c74fd68df26f-metrics-client-ca\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.501316 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.501287    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.501679 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.501660    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c47621d-ebe5-4469-a944-c74fd68df26f-node-exporter-tls\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.504308 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.504285    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:41.506806 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.506780    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kgvk\" (UniqueName: \"kubernetes.io/projected/1c47621d-ebe5-4469-a944-c74fd68df26f-kube-api-access-8kgvk\") pod \"node-exporter-bdfjd\" (UID: \"1c47621d-ebe5-4469-a944-c74fd68df26f\") " pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.597401 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.597376    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bdfjd"
Oct 30 18:36:41.644441 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:36:41.644414    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c47621d_ebe5_4469_a944_c74fd68df26f.slice/crio-f7f082ff9d64fe4c5bb2af4988905b5c077d4cedb34e82275b7e1db3c7e826ef WatchSource:0}: Error finding container f7f082ff9d64fe4c5bb2af4988905b5c077d4cedb34e82275b7e1db3c7e826ef: Status 404 returned error can't find the container with id f7f082ff9d64fe4c5bb2af4988905b5c077d4cedb34e82275b7e1db3c7e826ef
Oct 30 18:36:41.711002 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.710979    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:41.896262 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:41.896201    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:42.356825 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:42.356782    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:42.585533 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:42.585499    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:42.592387 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:42.592358    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bdfjd" event={"ID":"1c47621d-ebe5-4469-a944-c74fd68df26f","Type":"ContainerStarted","Data":"f7f082ff9d64fe4c5bb2af4988905b5c077d4cedb34e82275b7e1db3c7e826ef"}
Oct 30 18:36:43.298488 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:36:43.298455    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:36:43.541027 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:43.540998    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k6kkr"
Oct 30 18:36:43.597650 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:43.597623    4925 generic.go:356] "Generic (PLEG): container finished" podID="1c47621d-ebe5-4469-a944-c74fd68df26f" containerID="405deae6523d7d2baa032b13aa2157e654dd3225dfd8fb5d43d8a613eaaca992" exitCode=0
Oct 30 18:36:43.597784 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:43.597666    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bdfjd" event={"ID":"1c47621d-ebe5-4469-a944-c74fd68df26f","Type":"ContainerDied","Data":"405deae6523d7d2baa032b13aa2157e654dd3225dfd8fb5d43d8a613eaaca992"}
Oct 30 18:36:43.696540 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:43.696518    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:43.873726 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:43.873657    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:44.388772 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:44.388738    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:44.451892 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:44.451866    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:44.586815 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:44.586783    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:44.603777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:44.603750    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bdfjd" event={"ID":"1c47621d-ebe5-4469-a944-c74fd68df26f","Type":"ContainerStarted","Data":"9037881ad2e60af0f3fc99cb35b068774f8bc57c2fbc9ebcdabfdfcf356ed74e"}
Oct 30 18:36:44.603900 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:44.603792    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bdfjd" event={"ID":"1c47621d-ebe5-4469-a944-c74fd68df26f","Type":"ContainerStarted","Data":"0c3615271a06d75fa2c3542b7292993c4d1d02274bbe955197dd0d698ece737d"}
Oct 30 18:36:44.626174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:44.626076    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bdfjd" podStartSLOduration=2.124108739 podStartE2EDuration="3.626059506s" totalImagesPullingTime="1.414996471s" totalInitContainerRuntime="86.954296ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:44.62515897 +0000 UTC m=+26.341504050" watchObservedRunningTime="2026-10-30 18:36:44.626059506 +0000 UTC m=+26.342404583"
Oct 30 18:36:46.124300 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:46.124252    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:46.324657 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:46.324610    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:46.608561 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:46.608528    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:46.782008 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:46.781979    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:46.867282 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:46.867224    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:36:47.121635 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:47.121562    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:47.310369 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:47.310344    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:47.761836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:47.761803    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:48.187586 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:48.187551    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:48.366456 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:48.366431    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:51.094831 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:51.094796    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:51.290969 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:51.290943    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:55.764175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:55.764137    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:55.976488 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:55.976433    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:56.721690 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:56.721660    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6684c84797-p6964"]
Oct 30 18:36:56.926500 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:56.926467    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:57.104063 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:57.103988    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:58.590528 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:58.590444    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:36:59.467861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.467790    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76f45597c6-hfv8h" podUID="cd26f706-5971-4c47-a63e-699e7962ef8e" containerName="registry" containerID="cri-o://2026aceed699d170f6fb924c9495951809d1fba1c907a9a04102f6dd0dd7bd92" gracePeriod=30
Oct 30 18:36:59.671301 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.671270    4925 generic.go:356] "Generic (PLEG): container finished" podID="cd26f706-5971-4c47-a63e-699e7962ef8e" containerID="2026aceed699d170f6fb924c9495951809d1fba1c907a9a04102f6dd0dd7bd92" exitCode=0
Oct 30 18:36:59.671718 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.671324    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f45597c6-hfv8h" event={"ID":"cd26f706-5971-4c47-a63e-699e7962ef8e","Type":"ContainerDied","Data":"2026aceed699d170f6fb924c9495951809d1fba1c907a9a04102f6dd0dd7bd92"}
Oct 30 18:36:59.758652 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.758625    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:36:59.931413 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.931386    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-certificates\") pod \"cd26f706-5971-4c47-a63e-699e7962ef8e\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") "
Oct 30 18:36:59.931558 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.931435    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-kube-api-access-p9mdl\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-kube-api-access-p9mdl\") pod \"cd26f706-5971-4c47-a63e-699e7962ef8e\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") "
Oct 30 18:36:59.931558 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.931458    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-image-registry-private-configuration\") pod \"cd26f706-5971-4c47-a63e-699e7962ef8e\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") "
Oct 30 18:36:59.931558 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.931481    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-trusted-ca\") pod \"cd26f706-5971-4c47-a63e-699e7962ef8e\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") "
Oct 30 18:36:59.931703 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.931586    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd26f706-5971-4c47-a63e-699e7962ef8e-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd26f706-5971-4c47-a63e-699e7962ef8e-ca-trust-extracted\") pod \"cd26f706-5971-4c47-a63e-699e7962ef8e\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") "
Oct 30 18:36:59.931703 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.931665    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-bound-sa-token\") pod \"cd26f706-5971-4c47-a63e-699e7962ef8e\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") "
Oct 30 18:36:59.931804 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.931705    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") pod \"cd26f706-5971-4c47-a63e-699e7962ef8e\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") "
Oct 30 18:36:59.931804 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.931786    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-installation-pull-secrets\") pod \"cd26f706-5971-4c47-a63e-699e7962ef8e\" (UID: \"cd26f706-5971-4c47-a63e-699e7962ef8e\") "
Oct 30 18:36:59.931922 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.931893    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-trusted-ca" pod "cd26f706-5971-4c47-a63e-699e7962ef8e" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:36:59.932004 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.931979    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-certificates" pod "cd26f706-5971-4c47-a63e-699e7962ef8e" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:36:59.932124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.932081    4925 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-certificates\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:36:59.932196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.932132    4925 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd26f706-5971-4c47-a63e-699e7962ef8e-trusted-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:36:59.934206 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.934181    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-image-registry-private-configuration" pod "cd26f706-5971-4c47-a63e-699e7962ef8e" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:36:59.934426 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.934399    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-kube-api-access-p9mdl" pod "cd26f706-5971-4c47-a63e-699e7962ef8e" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e"). InnerVolumeSpecName "kube-api-access-p9mdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:36:59.934656 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.934622    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls" pod "cd26f706-5971-4c47-a63e-699e7962ef8e" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:36:59.934756 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.934738    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-bound-sa-token" pod "cd26f706-5971-4c47-a63e-699e7962ef8e" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:36:59.934843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.934826    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-installation-pull-secrets" pod "cd26f706-5971-4c47-a63e-699e7962ef8e" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:36:59.938936 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:36:59.938910    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd26f706-5971-4c47-a63e-699e7962ef8e-ca-trust-extracted" pod "cd26f706-5971-4c47-a63e-699e7962ef8e" (UID: "cd26f706-5971-4c47-a63e-699e7962ef8e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:37:00.033190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:00.033128    4925 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-bound-sa-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:00.033190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:00.033161    4925 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-registry-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:00.033190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:00.033171    4925 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-installation-pull-secrets\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:00.033190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:00.033182    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9mdl\" (UniqueName: \"kubernetes.io/projected/cd26f706-5971-4c47-a63e-699e7962ef8e-kube-api-access-p9mdl\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:00.033190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:00.033192    4925 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cd26f706-5971-4c47-a63e-699e7962ef8e-image-registry-private-configuration\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:00.033190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:00.033201    4925 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cd26f706-5971-4c47-a63e-699e7962ef8e-ca-trust-extracted\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:00.676615 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:00.676553    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76f45597c6-hfv8h"
Oct 30 18:37:00.676615 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:00.676570    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76f45597c6-hfv8h" event={"ID":"cd26f706-5971-4c47-a63e-699e7962ef8e","Type":"ContainerDied","Data":"45edf1de7c18daa21ebe78f3fe1ca7dfffc903f9c30b86fb40ac347f8b72a6d5"}
Oct 30 18:37:00.677027 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:00.676622    4925 scope.go:122] "RemoveContainer" containerID="2026aceed699d170f6fb924c9495951809d1fba1c907a9a04102f6dd0dd7bd92"
Oct 30 18:37:00.694370 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:00.694351    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76f45597c6-hfv8h"]
Oct 30 18:37:00.698175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:00.698155    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76f45597c6-hfv8h"]
Oct 30 18:37:02.419459 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:02.419427    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd26f706-5971-4c47-a63e-699e7962ef8e" path="/var/lib/kubelet/pods/cd26f706-5971-4c47-a63e-699e7962ef8e/volumes"
Oct 30 18:37:07.262594 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:07.262556    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:07.457136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:07.457070    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:07.820756 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:07.820716    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:08.133492 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:08.133417    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:09.219691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:09.219653    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:09.389234 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:09.389196    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:12.228216 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:12.228181    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:13.300948 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:37:13.300910    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:37:20.420292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:20.420251    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:20.609186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:20.609161    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:21.105175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:21.105121    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:21.287866 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:21.287829    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:21.745944 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:21.745906    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6684c84797-p6964" podUID="e32eb376-a16f-421c-8958-a410a5650964" containerName="registry" containerID="cri-o://a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e" gracePeriod=30
Oct 30 18:37:21.990314 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:21.990292    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:37:22.088002 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.087942    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-image-registry-private-configuration\") pod \"e32eb376-a16f-421c-8958-a410a5650964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") "
Oct 30 18:37:22.088002 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.087974    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-installation-pull-secrets\") pod \"e32eb376-a16f-421c-8958-a410a5650964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") "
Oct 30 18:37:22.088002 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.088000    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") pod \"e32eb376-a16f-421c-8958-a410a5650964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") "
Oct 30 18:37:22.088269 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.088031    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-registry-certificates\") pod \"e32eb376-a16f-421c-8958-a410a5650964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") "
Oct 30 18:37:22.088269 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.088058    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-kube-api-access-574hk\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-kube-api-access-574hk\") pod \"e32eb376-a16f-421c-8958-a410a5650964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") "
Oct 30 18:37:22.088269 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.088151    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e32eb376-a16f-421c-8958-a410a5650964-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e32eb376-a16f-421c-8958-a410a5650964-ca-trust-extracted\") pod \"e32eb376-a16f-421c-8958-a410a5650964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") "
Oct 30 18:37:22.088269 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.088199    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-bound-sa-token\") pod \"e32eb376-a16f-421c-8958-a410a5650964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") "
Oct 30 18:37:22.088269 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.088227    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-trusted-ca\") pod \"e32eb376-a16f-421c-8958-a410a5650964\" (UID: \"e32eb376-a16f-421c-8958-a410a5650964\") "
Oct 30 18:37:22.089061 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.088994    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-registry-certificates" pod "e32eb376-a16f-421c-8958-a410a5650964" (UID: "e32eb376-a16f-421c-8958-a410a5650964"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:37:22.089061 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.089004    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-trusted-ca" pod "e32eb376-a16f-421c-8958-a410a5650964" (UID: "e32eb376-a16f-421c-8958-a410a5650964"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:37:22.090645 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.090618    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-kube-api-access-574hk" pod "e32eb376-a16f-421c-8958-a410a5650964" (UID: "e32eb376-a16f-421c-8958-a410a5650964"). InnerVolumeSpecName "kube-api-access-574hk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:37:22.090748 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.090702    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-installation-pull-secrets" pod "e32eb376-a16f-421c-8958-a410a5650964" (UID: "e32eb376-a16f-421c-8958-a410a5650964"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:37:22.090894 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.090870    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls" pod "e32eb376-a16f-421c-8958-a410a5650964" (UID: "e32eb376-a16f-421c-8958-a410a5650964"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:37:22.091138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.091087    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-bound-sa-token" pod "e32eb376-a16f-421c-8958-a410a5650964" (UID: "e32eb376-a16f-421c-8958-a410a5650964"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:37:22.091214 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.091115    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-image-registry-private-configuration" pod "e32eb376-a16f-421c-8958-a410a5650964" (UID: "e32eb376-a16f-421c-8958-a410a5650964"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:37:22.095913 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.095879    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e32eb376-a16f-421c-8958-a410a5650964-ca-trust-extracted" pod "e32eb376-a16f-421c-8958-a410a5650964" (UID: "e32eb376-a16f-421c-8958-a410a5650964"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:37:22.188952 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.188924    4925 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-registry-certificates\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:22.188952 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.188951    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-574hk\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-kube-api-access-574hk\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:22.189122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.188966    4925 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e32eb376-a16f-421c-8958-a410a5650964-ca-trust-extracted\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:22.189122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.188981    4925 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-bound-sa-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:22.189122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.188994    4925 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e32eb376-a16f-421c-8958-a410a5650964-trusted-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:22.189122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.189008    4925 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-image-registry-private-configuration\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:22.189122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.189022    4925 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e32eb376-a16f-421c-8958-a410a5650964-installation-pull-secrets\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:22.189122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.189036    4925 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e32eb376-a16f-421c-8958-a410a5650964-registry-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:37:22.767796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.767733    4925 generic.go:356] "Generic (PLEG): container finished" podID="e32eb376-a16f-421c-8958-a410a5650964" containerID="a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e" exitCode=0
Oct 30 18:37:22.767796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.767793    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6684c84797-p6964"
Oct 30 18:37:22.768231 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.767824    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6684c84797-p6964" event={"ID":"e32eb376-a16f-421c-8958-a410a5650964","Type":"ContainerDied","Data":"a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e"}
Oct 30 18:37:22.768231 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.767882    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6684c84797-p6964" event={"ID":"e32eb376-a16f-421c-8958-a410a5650964","Type":"ContainerDied","Data":"e6324a28990980d094a8a00dead461a059a559fadf98846fb1a8611e8bc20ef0"}
Oct 30 18:37:22.768231 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.767906    4925 scope.go:122] "RemoveContainer" containerID="a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e"
Oct 30 18:37:22.776172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.776153    4925 scope.go:122] "RemoveContainer" containerID="a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e"
Oct 30 18:37:22.776507 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:37:22.776474    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e\": container with ID starting with a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e not found: ID does not exist" containerID="a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e"
Oct 30 18:37:22.776605 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.776512    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e"} err="failed to get container status \"a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e\": rpc error: code = NotFound desc = could not find container \"a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e\": container with ID starting with a89206f88798659793760c91e580907d014a1a1b84baa8b0d5da531974b3424e not found: ID does not exist"
Oct 30 18:37:22.785918 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.785895    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6684c84797-p6964"]
Oct 30 18:37:22.791337 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:22.791315    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6684c84797-p6964"]
Oct 30 18:37:23.774824 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:23.774793    4925 generic.go:356] "Generic (PLEG): container finished" podID="4bee5094-7886-4fa9-aa7c-0ddc3d518619" containerID="9f316231f546c805d905e27b7a37bafbaf4997ac7e8ff8fc9f3be227e1e2f16e" exitCode=0
Oct 30 18:37:23.775231 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:23.774876    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc" event={"ID":"4bee5094-7886-4fa9-aa7c-0ddc3d518619","Type":"ContainerDied","Data":"9f316231f546c805d905e27b7a37bafbaf4997ac7e8ff8fc9f3be227e1e2f16e"}
Oct 30 18:37:23.775272 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:23.775228    4925 scope.go:122] "RemoveContainer" containerID="9f316231f546c805d905e27b7a37bafbaf4997ac7e8ff8fc9f3be227e1e2f16e"
Oct 30 18:37:24.418925 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:24.418894    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e32eb376-a16f-421c-8958-a410a5650964" path="/var/lib/kubelet/pods/e32eb376-a16f-421c-8958-a410a5650964/volumes"
Oct 30 18:37:24.779686 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:24.779606    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-m7bvc" event={"ID":"4bee5094-7886-4fa9-aa7c-0ddc3d518619","Type":"ContainerStarted","Data":"cc55d6bf12346de9fe97a6c5e769cc0e5568e86f4e30bb5a61e248878fb76021"}
Oct 30 18:37:28.795664 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:28.795631    4925 generic.go:356] "Generic (PLEG): container finished" podID="d2b52b9e-4797-4a2e-953e-7da0f61bca87" containerID="fef06f521cefc21fc9428e432194bfddcf361a1b2aa60bd9338d92cb92d590ee" exitCode=0
Oct 30 18:37:28.796177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:28.795713    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg" event={"ID":"d2b52b9e-4797-4a2e-953e-7da0f61bca87","Type":"ContainerDied","Data":"fef06f521cefc21fc9428e432194bfddcf361a1b2aa60bd9338d92cb92d590ee"}
Oct 30 18:37:28.796177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:28.796055    4925 scope.go:122] "RemoveContainer" containerID="fef06f521cefc21fc9428e432194bfddcf361a1b2aa60bd9338d92cb92d590ee"
Oct 30 18:37:29.801225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:29.801187    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-lhwcg" event={"ID":"d2b52b9e-4797-4a2e-953e-7da0f61bca87","Type":"ContainerStarted","Data":"06c52fe1d76ed865b941c7deeb9a5e10a95ae84f72aad8803df42d2e20de6e63"}
Oct 30 18:37:29.920831 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:29.920805    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:30.723042 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:30.723005    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:30.916686 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:30.916647    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:33.666505 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:33.666474    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:33.818621 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:33.818590    4925 generic.go:356] "Generic (PLEG): container finished" podID="0b90705d-6abb-481d-8af0-89e542709329" containerID="8f87af716388ad7b5d0e3079975706e1c5fd8938e44cee25b1b8ed342a6e83d1" exitCode=0
Oct 30 18:37:33.818762 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:33.818629    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-qhk9m" event={"ID":"0b90705d-6abb-481d-8af0-89e542709329","Type":"ContainerDied","Data":"8f87af716388ad7b5d0e3079975706e1c5fd8938e44cee25b1b8ed342a6e83d1"}
Oct 30 18:37:33.818914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:33.818899    4925 scope.go:122] "RemoveContainer" containerID="8f87af716388ad7b5d0e3079975706e1c5fd8938e44cee25b1b8ed342a6e83d1"
Oct 30 18:37:33.847779 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:33.847736    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:34.828807 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:34.828767    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-qhk9m" event={"ID":"0b90705d-6abb-481d-8af0-89e542709329","Type":"ContainerStarted","Data":"01ef0540016d6e90fbe003ecc7e8495a253e87d6883f42a279f74103eb3ac85c"}
Oct 30 18:37:36.089079 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:36.089038    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:36.286185 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:37:36.286141    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:43.301585 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:37:43.301542    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:38:07.885922 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:38:07.885887    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:10.655214 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:38:10.655178    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:10.952038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:38:10.952006    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:11.352439 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:38:11.352366    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:13.301408 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:38:13.301363    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:38:15.426712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:38:15.426664    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:18.556581 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:38:18.556549    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:18.929303 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:38:18.929270    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:19.127201 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:38:19.127147    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:43.301915 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:38:43.301875    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:39:00.705749 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:00.705706    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:00.915845 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:00.915816    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:07.907447 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:07.907414    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:13.300211 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:39:13.300171    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:39:26.009322 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:26.009279    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:26.189678 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:26.189637    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:31.171614 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:31.171578    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:32.905184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:32.905119    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:33.080105 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:33.080069    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:34.321774 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:34.321742    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:34.511748 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:34.511690    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:38.777734 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:38.777690    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:38.948789 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:38.948741    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:43.302074 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:39:43.302036    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:39:45.138374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:45.138334    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:45.307917 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:39:45.307874    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:07.970008 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:40:07.969914    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:13.301671 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:40:13.301633    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:40:39.962382 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:40:39.962340    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:43.303190 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:40:43.303145    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:41:08.007816 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:41:08.007779    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:41:13.300766 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:41:13.300730    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:41:18.432716 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:41:18.432683    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 18:41:18.433341 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:41:18.433323    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 18:41:43.303493 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:41:43.303441    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:41:52.837446 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:41:52.837409    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:05.460877 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:05.460836    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:05.723245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:05.723146    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:06.960707 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.960664    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"]
Oct 30 18:42:06.961117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.960972    4925 cpu_manager.go:414] "containerMap: removing container" podUID="e32eb376-a16f-421c-8958-a410a5650964" containerName="registry"
Oct 30 18:42:06.961117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.960983    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e32eb376-a16f-421c-8958-a410a5650964" containerName="registry"
Oct 30 18:42:06.961117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.961005    4925 cpu_manager.go:414] "containerMap: removing container" podUID="cd26f706-5971-4c47-a63e-699e7962ef8e" containerName="registry"
Oct 30 18:42:06.961117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.961021    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd26f706-5971-4c47-a63e-699e7962ef8e" containerName="registry"
Oct 30 18:42:06.961117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.961066    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd26f706-5971-4c47-a63e-699e7962ef8e" containerName="registry"
Oct 30 18:42:06.961117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.961076    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="e32eb376-a16f-421c-8958-a410a5650964" containerName="registry"
Oct 30 18:42:06.964006 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.963991    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:06.967362 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.967334    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 18:42:06.967486 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.967386    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 18:42:06.967486 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.967386    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 18:42:06.967486 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.967415    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 18:42:06.967486 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.967439    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dqnb2\""
Oct 30 18:42:06.975524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.975504    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"]
Oct 30 18:42:06.991830 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.991807    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:06.991934 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.991861    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:06.991990 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.991973    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:06.992036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.992006    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:06.992140 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.992123    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:06.992183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.992155    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:06.992183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:06.992175    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9x4w\" (UniqueName: \"kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-kube-api-access-b9x4w\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.069950 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.069900    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:07.093011 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.092988    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.093082 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.093024    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.093082 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.093052    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.093082 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.093077    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.093239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.093147    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.093239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.093183    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.093239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.093205    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9x4w\" (UniqueName: \"kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-kube-api-access-b9x4w\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.093673 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.093645    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.095582 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.095558    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.095684 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.095634    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.095876 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.095855    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.095987 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.095971    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.101202 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.101183    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.101757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.101734    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9x4w\" (UniqueName: \"kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-kube-api-access-b9x4w\") pod \"istiod-openshift-gateway-7876bfcb74-nqdlt\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.258031 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.257972    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:07.275748 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.275724    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:07.554883 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.554806    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"]
Oct 30 18:42:07.558422 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:42:07.558391    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc148d7b6_dbe2_4e1c_8b25_d9f5c613f6f5.slice/crio-f3c54efbfe239e25b014797c7563a339729bb1a0d433109e57cf0568b9bdf23b WatchSource:0}: Error finding container f3c54efbfe239e25b014797c7563a339729bb1a0d433109e57cf0568b9bdf23b: Status 404 returned error can't find the container with id f3c54efbfe239e25b014797c7563a339729bb1a0d433109e57cf0568b9bdf23b
Oct 30 18:42:07.560922 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.560895    4925 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:42:07.818384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:07.818304    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt" event={"ID":"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5","Type":"ContainerStarted","Data":"f3c54efbfe239e25b014797c7563a339729bb1a0d433109e57cf0568b9bdf23b"}
Oct 30 18:42:08.051929 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:08.051896    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:08.239219 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:08.239181    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:10.319961 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:10.319929    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:42:10.320235 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:10.319995    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:42:10.791085 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:10.791052    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:10.834400 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:10.834370    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt" event={"ID":"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5","Type":"ContainerStarted","Data":"55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b"}
Oct 30 18:42:10.834544 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:10.834427    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:10.860073 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:10.860033    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt" podStartSLOduration=2.102002138 podStartE2EDuration="4.860020476s" totalImagesPullingTime="2.758018338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:10.858022832 +0000 UTC m=+352.574367909" watchObservedRunningTime="2026-10-30 18:42:10.860020476 +0000 UTC m=+352.576365551"
Oct 30 18:42:10.981338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:10.981310    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:11.532812 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:11.532783    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:11.718348 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:11.718269    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:11.840784 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:11.840746    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:42:13.300368 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:42:13.300335    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:42:14.211005 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:14.210971    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:43.299834 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:42:43.299800    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:42:45.468583 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:45.468536    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:49.288332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:49.288304    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:49.494686 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:49.494646    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:49.948322 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:49.948290    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:50.006494 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.006466    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"]
Oct 30 18:42:50.015002 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.014981    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.049329 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.049306    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"]
Oct 30 18:42:50.082302 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.082274    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 18:42:50.082302 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.082296    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 18:42:50.082447 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.082408    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 18:42:50.082505 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.082488    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 18:42:50.082560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.082546    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-2vxg7\""
Oct 30 18:42:50.082602 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.082589    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 18:42:50.091874 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.091842    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-metrics-cert\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.091947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.091894    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-tmp\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.091998 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.091984    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-webhook-cert\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.092034 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.092016    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv9z8\" (UniqueName: \"kubernetes.io/projected/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-kube-api-access-mv9z8\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.092071 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.092045    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-manifests\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.092156 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.092131    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.193437 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.193412    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-tmp\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.193546 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.193463    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-webhook-cert\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.193546 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.193490    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv9z8\" (UniqueName: \"kubernetes.io/projected/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-kube-api-access-mv9z8\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.193546 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.193516    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-manifests\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.193712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.193611    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.193712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.193647    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-metrics-cert\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.193911 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.193891    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-manifests\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.196008 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.195981    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-webhook-cert\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.196122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.195982    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-tmp\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.196122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.196024    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.196201 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.196175    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-metrics-cert\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.201937 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.201890    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv9z8\" (UniqueName: \"kubernetes.io/projected/fd87c2d3-e86a-41be-8690-8d3b09fc8dba-kube-api-access-mv9z8\") pod \"opendatahub-operator-controller-manager-8ff44d5d-qk6hc\" (UID: \"fd87c2d3-e86a-41be-8690-8d3b09fc8dba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.280282 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.280254    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:50.330287 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.330266    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:50.464946 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.464882    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:50.653499 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.653464    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"]
Oct 30 18:42:50.654059 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:42:50.654027    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd87c2d3_e86a_41be_8690_8d3b09fc8dba.slice/crio-d2dc2d44da833e8cf7016a89bfc1ccca1e087aaf2feaf06c97158c7709dc8b76 WatchSource:0}: Error finding container d2dc2d44da833e8cf7016a89bfc1ccca1e087aaf2feaf06c97158c7709dc8b76: Status 404 returned error can't find the container with id d2dc2d44da833e8cf7016a89bfc1ccca1e087aaf2feaf06c97158c7709dc8b76
Oct 30 18:42:50.717623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.717572    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:50.914046 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:50.914013    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:51.004304 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:51.004240    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc" event={"ID":"fd87c2d3-e86a-41be-8690-8d3b09fc8dba","Type":"ContainerStarted","Data":"d2dc2d44da833e8cf7016a89bfc1ccca1e087aaf2feaf06c97158c7709dc8b76"}
Oct 30 18:42:53.091368 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:53.091332    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:53.287911 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:53.287861    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:53.464777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:53.464736    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:54.017949 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:54.017887    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc" event={"ID":"fd87c2d3-e86a-41be-8690-8d3b09fc8dba","Type":"ContainerStarted","Data":"3acc302c965768638ed3775912ff47f89bc82a0d0d803b457a33511da2934730"}
Oct 30 18:42:55.024062 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:55.024019    4925 generic.go:356] "Generic (PLEG): container finished" podID="fd87c2d3-e86a-41be-8690-8d3b09fc8dba" containerID="3acc302c965768638ed3775912ff47f89bc82a0d0d803b457a33511da2934730" exitCode=0
Oct 30 18:42:55.024516 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:55.024078    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc" event={"ID":"fd87c2d3-e86a-41be-8690-8d3b09fc8dba","Type":"ContainerDied","Data":"3acc302c965768638ed3775912ff47f89bc82a0d0d803b457a33511da2934730"}
Oct 30 18:42:56.030046 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:56.030012    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc" event={"ID":"fd87c2d3-e86a-41be-8690-8d3b09fc8dba","Type":"ContainerStarted","Data":"1211cdfaa01d8c7e443b664d9bf408e4ea0ced5da66bfb83665be0a5311f8faf"}
Oct 30 18:42:56.030467 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:56.030151    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:42:56.086733 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:56.086673    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc" podStartSLOduration=3.138667276 podStartE2EDuration="7.086654059s" totalImagesPullingTime="3.53663123s" totalInitContainerRuntime="411.355553ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:42:56.08480524 +0000 UTC m=+397.801150317" watchObservedRunningTime="2026-10-30 18:42:56.086654059 +0000 UTC m=+397.802999135"
Oct 30 18:42:57.933860 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:57.933814    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:58.105065 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:42:58.105029    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:01.302018 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:01.301928    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:02.560634 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:02.560603    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:03.505475 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:03.505444    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:04.766015 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:04.765972    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:07.036451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:07.036421    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8ff44d5d-qk6hc"
Oct 30 18:43:07.359991 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:07.359903    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:08.144736 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:08.144682    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:10.350371 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:10.350330    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:12.290821 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:12.290787    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:12.658791 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:12.658761    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:12.888049 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:12.888013    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:13.301903 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:43:13.301865    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:43:14.890572 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:14.890530    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:17.076370 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:17.076331    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:17.305488 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:17.305453    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:17.726234 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:17.726197    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:17.955678 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:17.955643    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:18.090139 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.090045    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"]
Oct 30 18:43:18.094998 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.094978    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"
Oct 30 18:43:18.105720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.105703    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 18:43:18.106150 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.106130    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 18:43:18.106150 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.106133    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-nzddx\""
Oct 30 18:43:18.113463 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.113444    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"]
Oct 30 18:43:18.201758 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.201729    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44bst\" (UniqueName: \"kubernetes.io/projected/24691fdd-6f67-43e4-bd56-446e31b2c0a2-kube-api-access-44bst\") pod \"kube-auth-proxy-6f89dc687f-bwx2f\" (UID: \"24691fdd-6f67-43e4-bd56-446e31b2c0a2\") " pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"
Oct 30 18:43:18.201758 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.201758    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24691fdd-6f67-43e4-bd56-446e31b2c0a2-tmp\") pod \"kube-auth-proxy-6f89dc687f-bwx2f\" (UID: \"24691fdd-6f67-43e4-bd56-446e31b2c0a2\") " pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"
Oct 30 18:43:18.201962 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.201855    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24691fdd-6f67-43e4-bd56-446e31b2c0a2-tls-certs\") pod \"kube-auth-proxy-6f89dc687f-bwx2f\" (UID: \"24691fdd-6f67-43e4-bd56-446e31b2c0a2\") " pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"
Oct 30 18:43:18.244702 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.244653    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:18.302793 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.302772    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24691fdd-6f67-43e4-bd56-446e31b2c0a2-tls-certs\") pod \"kube-auth-proxy-6f89dc687f-bwx2f\" (UID: \"24691fdd-6f67-43e4-bd56-446e31b2c0a2\") " pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"
Oct 30 18:43:18.302905 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.302809    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44bst\" (UniqueName: \"kubernetes.io/projected/24691fdd-6f67-43e4-bd56-446e31b2c0a2-kube-api-access-44bst\") pod \"kube-auth-proxy-6f89dc687f-bwx2f\" (UID: \"24691fdd-6f67-43e4-bd56-446e31b2c0a2\") " pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"
Oct 30 18:43:18.302905 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.302829    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24691fdd-6f67-43e4-bd56-446e31b2c0a2-tmp\") pod \"kube-auth-proxy-6f89dc687f-bwx2f\" (UID: \"24691fdd-6f67-43e4-bd56-446e31b2c0a2\") " pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"
Oct 30 18:43:18.305303 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.305280    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24691fdd-6f67-43e4-bd56-446e31b2c0a2-tmp\") pod \"kube-auth-proxy-6f89dc687f-bwx2f\" (UID: \"24691fdd-6f67-43e4-bd56-446e31b2c0a2\") " pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"
Oct 30 18:43:18.305469 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.305447    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24691fdd-6f67-43e4-bd56-446e31b2c0a2-tls-certs\") pod \"kube-auth-proxy-6f89dc687f-bwx2f\" (UID: \"24691fdd-6f67-43e4-bd56-446e31b2c0a2\") " pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"
Oct 30 18:43:18.336899 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.336876    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44bst\" (UniqueName: \"kubernetes.io/projected/24691fdd-6f67-43e4-bd56-446e31b2c0a2-kube-api-access-44bst\") pod \"kube-auth-proxy-6f89dc687f-bwx2f\" (UID: \"24691fdd-6f67-43e4-bd56-446e31b2c0a2\") " pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"
Oct 30 18:43:18.409626 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.409607    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-nzddx\""
Oct 30 18:43:18.417259 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.417241    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"
Oct 30 18:43:18.503264 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.502718    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:18.702855 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.702778    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f"]
Oct 30 18:43:18.706154 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:43:18.706077    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24691fdd_6f67_43e4_bd56_446e31b2c0a2.slice/crio-655e303f1390e392f38a65e9643f401b88075be0a40558e1720d5605cd6c87d4 WatchSource:0}: Error finding container 655e303f1390e392f38a65e9643f401b88075be0a40558e1720d5605cd6c87d4: Status 404 returned error can't find the container with id 655e303f1390e392f38a65e9643f401b88075be0a40558e1720d5605cd6c87d4
Oct 30 18:43:18.715282 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.715257    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:18.963399 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:18.963333    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:19.132902 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:19.132875    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f" event={"ID":"24691fdd-6f67-43e4-bd56-446e31b2c0a2","Type":"ContainerStarted","Data":"655e303f1390e392f38a65e9643f401b88075be0a40558e1720d5605cd6c87d4"}
Oct 30 18:43:19.228963 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:19.228904    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:19.564044 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:19.563885    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:20.228707 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:20.228661    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:20.501572 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:20.501468    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:20.701408 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:20.701378    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:20.943580 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:20.943541    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:21.060723 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:21.060704    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 18:43:21.143724 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:21.143696    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f" event={"ID":"24691fdd-6f67-43e4-bd56-446e31b2c0a2","Type":"ContainerStarted","Data":"25c79fdcf7bd36c7a1122e593aad658cf752a60e48e2b8161168bc03d19a615d"}
Oct 30 18:43:21.167532 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:21.167482    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6f89dc687f-bwx2f" podStartSLOduration=0.817512351 podStartE2EDuration="3.167465447s" totalImagesPullingTime="2.349953096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:21.166475477 +0000 UTC m=+422.882820553" watchObservedRunningTime="2026-10-30 18:43:21.167465447 +0000 UTC m=+422.883810524"
Oct 30 18:43:24.495075 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:24.495035    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:24.675720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:24.675684    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:25.317942 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:25.317900    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:25.506882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:25.506852    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:25.723770 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:25.723743    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:25.935140 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:25.935077    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:26.896758 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:26.896716    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"]
Oct 30 18:43:26.903671 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:26.903650    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"
Oct 30 18:43:26.920670 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:26.920648    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 18:43:26.921674 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:26.921653    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-l8lft\""
Oct 30 18:43:26.921759 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:26.921750    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 18:43:26.923028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:26.923007    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"]
Oct 30 18:43:27.074573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:27.074551    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43fb2f5e-93e5-4aef-8f44-1aefbc65d567-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-hg8tr\" (UID: \"43fb2f5e-93e5-4aef-8f44-1aefbc65d567\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"
Oct 30 18:43:27.074695 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:27.074583    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hxrq\" (UniqueName: \"kubernetes.io/projected/43fb2f5e-93e5-4aef-8f44-1aefbc65d567-kube-api-access-7hxrq\") pod \"kuadrant-operator-controller-manager-678d87766f-hg8tr\" (UID: \"43fb2f5e-93e5-4aef-8f44-1aefbc65d567\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"
Oct 30 18:43:27.175337 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:27.175309    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43fb2f5e-93e5-4aef-8f44-1aefbc65d567-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-hg8tr\" (UID: \"43fb2f5e-93e5-4aef-8f44-1aefbc65d567\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"
Oct 30 18:43:27.175469 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:27.175348    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hxrq\" (UniqueName: \"kubernetes.io/projected/43fb2f5e-93e5-4aef-8f44-1aefbc65d567-kube-api-access-7hxrq\") pod \"kuadrant-operator-controller-manager-678d87766f-hg8tr\" (UID: \"43fb2f5e-93e5-4aef-8f44-1aefbc65d567\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"
Oct 30 18:43:27.175677 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:27.175653    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43fb2f5e-93e5-4aef-8f44-1aefbc65d567-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-hg8tr\" (UID: \"43fb2f5e-93e5-4aef-8f44-1aefbc65d567\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"
Oct 30 18:43:27.185644 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:27.185622    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hxrq\" (UniqueName: \"kubernetes.io/projected/43fb2f5e-93e5-4aef-8f44-1aefbc65d567-kube-api-access-7hxrq\") pod \"kuadrant-operator-controller-manager-678d87766f-hg8tr\" (UID: \"43fb2f5e-93e5-4aef-8f44-1aefbc65d567\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"
Oct 30 18:43:27.215036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:27.215012    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"
Oct 30 18:43:27.547621 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:27.547574    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"]
Oct 30 18:43:27.550042 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:43:27.550015    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43fb2f5e_93e5_4aef_8f44_1aefbc65d567.slice/crio-431831f20778910967f0699d9c5f641dea1d261d8bec385d2b141dca23c8e3d4 WatchSource:0}: Error finding container 431831f20778910967f0699d9c5f641dea1d261d8bec385d2b141dca23c8e3d4: Status 404 returned error can't find the container with id 431831f20778910967f0699d9c5f641dea1d261d8bec385d2b141dca23c8e3d4
Oct 30 18:43:28.174845 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:28.174812    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr" event={"ID":"43fb2f5e-93e5-4aef-8f44-1aefbc65d567","Type":"ContainerStarted","Data":"431831f20778910967f0699d9c5f641dea1d261d8bec385d2b141dca23c8e3d4"}
Oct 30 18:43:31.083005 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:31.082968    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:31.264987 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:31.264951    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:32.196418 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:32.196341    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr" event={"ID":"43fb2f5e-93e5-4aef-8f44-1aefbc65d567","Type":"ContainerStarted","Data":"4f24da6c021bb0ed010d0f59a9ecb8f340ee00d0bfc70b64711f45e15b6173b0"}
Oct 30 18:43:32.196826 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:32.196517    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"
Oct 30 18:43:32.237351 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:32.237306    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr" podStartSLOduration=1.85298689 podStartE2EDuration="6.237293993s" totalImagesPullingTime="4.384307103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:32.235693359 +0000 UTC m=+433.952038446" watchObservedRunningTime="2026-10-30 18:43:32.237293993 +0000 UTC m=+433.953639063"
Oct 30 18:43:33.914015 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:33.913981    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:34.107319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:34.107286    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:36.352788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:36.352747    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:36.524987 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:36.524957    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:41.105663 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:41.105625    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:41.281469 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:41.281429    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:43.203535 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:43.203498    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-hg8tr"
Oct 30 18:43:43.303632 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:43:43.303591    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:43:44.482581 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:44.482550    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:44.661902 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:44.661865    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:45.944340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:45.944309    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:46.124677 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:46.124639    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:54.785054 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:54.785012    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:54.968436 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:54.968399    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:57.222452 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:57.222414    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:57.391935 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:57.391905    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:59.421963 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:43:59.421932    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:00.525938 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:00.525903    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:00.698762 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:00.698726    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:01.262439 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:01.262410    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:01.446475 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:01.446436    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:01.978352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:01.978322    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:02.158500 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:02.158464    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:06.995663 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:06.995626    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:07.176219 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:07.176187    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:07.383151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:07.383038    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:07.616114 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:07.616060    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:07.822913 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:07.822881    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:08.214040 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:08.214003    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:08.423506 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:08.423473    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:13.299493 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:44:13.299457    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:44:14.109455 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.109418    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:14.315111 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.315062    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:14.454997 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.454968    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"]
Oct 30 18:44:14.458984 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.458964    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.471025 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.471000    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"]
Oct 30 18:44:14.505598 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.505569    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:14.542792 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.542766    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad2e18b3-d6e4-468b-8f40-6e418841774e-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.542899 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.542797    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ad2e18b3-d6e4-468b-8f40-6e418841774e-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.542899 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.542816    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ad2e18b3-d6e4-468b-8f40-6e418841774e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.543013 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.542938    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ad2e18b3-d6e4-468b-8f40-6e418841774e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.543013 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.542972    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj5wb\" (UniqueName: \"kubernetes.io/projected/ad2e18b3-d6e4-468b-8f40-6e418841774e-kube-api-access-rj5wb\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.543133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.543066    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ad2e18b3-d6e4-468b-8f40-6e418841774e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.543133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.543117    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad2e18b3-d6e4-468b-8f40-6e418841774e-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.643770 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.643743    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ad2e18b3-d6e4-468b-8f40-6e418841774e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.643851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.643772    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj5wb\" (UniqueName: \"kubernetes.io/projected/ad2e18b3-d6e4-468b-8f40-6e418841774e-kube-api-access-rj5wb\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.643851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.643807    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ad2e18b3-d6e4-468b-8f40-6e418841774e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.643851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.643834    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad2e18b3-d6e4-468b-8f40-6e418841774e-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.643980 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.643874    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad2e18b3-d6e4-468b-8f40-6e418841774e-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.643980 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.643906    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ad2e18b3-d6e4-468b-8f40-6e418841774e-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.643980 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.643929    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ad2e18b3-d6e4-468b-8f40-6e418841774e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.644601 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.644574    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ad2e18b3-d6e4-468b-8f40-6e418841774e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.647076 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.647048    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad2e18b3-d6e4-468b-8f40-6e418841774e-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.647244 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.647224    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ad2e18b3-d6e4-468b-8f40-6e418841774e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.647330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.647309    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ad2e18b3-d6e4-468b-8f40-6e418841774e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.647389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.647317    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ad2e18b3-d6e4-468b-8f40-6e418841774e-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.653643 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.653623    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad2e18b3-d6e4-468b-8f40-6e418841774e-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.653926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.653907    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj5wb\" (UniqueName: \"kubernetes.io/projected/ad2e18b3-d6e4-468b-8f40-6e418841774e-kube-api-access-rj5wb\") pod \"istiod-openshift-gateway-7ddd9cfdc8-rqh7r\" (UID: \"ad2e18b3-d6e4-468b-8f40-6e418841774e\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:14.769768 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:14.769718    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:15.046794 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:15.046728    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"]
Oct 30 18:44:15.050356 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:44:15.050330    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad2e18b3_d6e4_468b_8f40_6e418841774e.slice/crio-34b548f710de680fe3caf1dede8ceede57e2674c89c7e486542e052f37c02ab8 WatchSource:0}: Error finding container 34b548f710de680fe3caf1dede8ceede57e2674c89c7e486542e052f37c02ab8: Status 404 returned error can't find the container with id 34b548f710de680fe3caf1dede8ceede57e2674c89c7e486542e052f37c02ab8
Oct 30 18:44:15.052457 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:15.052413    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:44:15.052545 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:15.052492    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:44:15.378584 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:15.378502    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r" event={"ID":"ad2e18b3-d6e4-468b-8f40-6e418841774e","Type":"ContainerStarted","Data":"669e237de92ac954cf7995cd5f02b70e5a01e2bdb9cebfe5df5a8deefe230de6"}
Oct 30 18:44:15.379069 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:15.379040    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:15.379069 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:15.379068    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r" event={"ID":"ad2e18b3-d6e4-468b-8f40-6e418841774e","Type":"ContainerStarted","Data":"34b548f710de680fe3caf1dede8ceede57e2674c89c7e486542e052f37c02ab8"}
Oct 30 18:44:15.434467 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:15.434418    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r" podStartSLOduration=1.434405479 podStartE2EDuration="1.434405479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:15.431777136 +0000 UTC m=+477.148122213" watchObservedRunningTime="2026-10-30 18:44:15.434405479 +0000 UTC m=+477.150750556"
Oct 30 18:44:15.551495 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:15.551453    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:15.754878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:15.754847    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:16.186268 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.186231    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:16.375138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.375079    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:16.384391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.384370    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rqh7r"
Oct 30 18:44:16.482012 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.481946    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"]
Oct 30 18:44:16.482172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.482138    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt" podUID="c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" containerName="discovery" containerID="cri-o://55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b" gracePeriod=30
Oct 30 18:44:16.677600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.677571    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:16.719372 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.719353    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:44:16.762337 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.762268    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-kube-api-access-b9x4w\" (UniqueName: \"kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-kube-api-access-b9x4w\") pod \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") "
Oct 30 18:44:16.762337 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.762310    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-local-certs\") pod \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") "
Oct 30 18:44:16.762514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.762355    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-token\" (UniqueName: \"kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-token\") pod \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") "
Oct 30 18:44:16.762514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.762374    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-dns-cert\") pod \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") "
Oct 30 18:44:16.762514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.762394    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-cacerts\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-cacerts\") pod \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") "
Oct 30 18:44:16.762514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.762454    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-ca-configmap\") pod \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") "
Oct 30 18:44:16.762514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.762492    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-kubeconfig\") pod \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\" (UID: \"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5\") "
Oct 30 18:44:16.763228 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.763181    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-ca-configmap" pod "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" (UID: "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:44:16.764955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.764932    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-token" pod "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" (UID: "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:44:16.765358 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.765331    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-local-certs" pod "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" (UID: "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:44:16.765455 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.765388    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-kube-api-access-b9x4w" pod "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" (UID: "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5"). InnerVolumeSpecName "kube-api-access-b9x4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:44:16.765455 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.765393    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-dns-cert" pod "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" (UID: "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:44:16.765455 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.765436    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-cacerts" pod "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" (UID: "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:44:16.766385 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.766367    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-kubeconfig" pod "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" (UID: "c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:44:16.863197 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.863168    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9x4w\" (UniqueName: \"kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-kube-api-access-b9x4w\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:16.863197 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.863194    4925 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-local-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:16.863343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.863208    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:16.863343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.863221    4925 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-dns-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:16.863343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.863233    4925 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-cacerts\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:16.863343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.863246    4925 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-csr-ca-configmap\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:16.863343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.863259    4925 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5-istio-kubeconfig\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:16.864338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:16.864302    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:17.388809 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:17.388777    4925 generic.go:356] "Generic (PLEG): container finished" podID="c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" containerID="55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b" exitCode=0
Oct 30 18:44:17.389316 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:17.388842    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"
Oct 30 18:44:17.389316 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:17.388865    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt" event={"ID":"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5","Type":"ContainerDied","Data":"55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b"}
Oct 30 18:44:17.389316 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:17.388919    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt" event={"ID":"c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5","Type":"ContainerDied","Data":"f3c54efbfe239e25b014797c7563a339729bb1a0d433109e57cf0568b9bdf23b"}
Oct 30 18:44:17.389316 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:17.388939    4925 scope.go:122] "RemoveContainer" containerID="55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b"
Oct 30 18:44:17.398070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:17.398053    4925 scope.go:122] "RemoveContainer" containerID="55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b"
Oct 30 18:44:17.398396 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:44:17.398376    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b\": container with ID starting with 55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b not found: ID does not exist" containerID="55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b"
Oct 30 18:44:17.398467 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:17.398406    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b"} err="failed to get container status \"55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b\": rpc error: code = NotFound desc = could not find container \"55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b\": container with ID starting with 55704fa5020f36c19a4df4f6784d78dd5d13c47ff8021bcf01999f0f4614885b not found: ID does not exist"
Oct 30 18:44:17.420132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:17.420085    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"]
Oct 30 18:44:17.426768 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:17.426746    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-nqdlt"]
Oct 30 18:44:17.529836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:17.529817    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:17.752194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:17.752167    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:18.424606 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:18.424567    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" path="/var/lib/kubelet/pods/c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5/volumes"
Oct 30 18:44:18.643581 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:18.643552    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:18.854936 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:18.854854    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:23.400849 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:23.400796    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:23.582428 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:23.582394    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:23.878669 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:23.878637    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:24.064225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:24.064195    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:36.818615 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.818575    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5dfc4f849f-pjjzp"]
Oct 30 18:44:36.819072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.818899    4925 cpu_manager.go:414] "containerMap: removing container" podUID="c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" containerName="discovery"
Oct 30 18:44:36.819072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.818914    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" containerName="discovery"
Oct 30 18:44:36.819072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.818989    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="c148d7b6-dbe2-4e1c-8b25-d9f5c613f6f5" containerName="discovery"
Oct 30 18:44:36.820762 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.820743    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:36.825843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.825822    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 18:44:36.826876 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.826850    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 18:44:36.827128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.827111    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 18:44:36.827877 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.827861    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 18:44:36.828207 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.828191    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7f4zk\""
Oct 30 18:44:36.828272 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.828211    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 18:44:36.835324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.835304    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5dfc4f849f-pjjzp"]
Oct 30 18:44:36.868313 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.868278    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:36.908235 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.908209    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-webhook-cert\") pod \"maas-controller-5dfc4f849f-pjjzp\" (UID: \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\") " pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:36.908357 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.908277    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75cs6\" (UniqueName: \"kubernetes.io/projected/e19aae86-83f1-406a-9cbf-7b46b11fcb00-kube-api-access-75cs6\") pod \"maas-controller-5dfc4f849f-pjjzp\" (UID: \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\") " pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:36.908426 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:36.908385    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-metrics-certs\") pod \"maas-controller-5dfc4f849f-pjjzp\" (UID: \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\") " pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:37.009432 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.009400    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75cs6\" (UniqueName: \"kubernetes.io/projected/e19aae86-83f1-406a-9cbf-7b46b11fcb00-kube-api-access-75cs6\") pod \"maas-controller-5dfc4f849f-pjjzp\" (UID: \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\") " pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:37.009543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.009451    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-metrics-certs\") pod \"maas-controller-5dfc4f849f-pjjzp\" (UID: \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\") " pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:37.009543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.009483    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-webhook-cert\") pod \"maas-controller-5dfc4f849f-pjjzp\" (UID: \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\") " pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:37.012204 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.012182    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-webhook-cert\") pod \"maas-controller-5dfc4f849f-pjjzp\" (UID: \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\") " pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:37.012297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.012252    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-metrics-certs\") pod \"maas-controller-5dfc4f849f-pjjzp\" (UID: \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\") " pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:37.016794 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.016771    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75cs6\" (UniqueName: \"kubernetes.io/projected/e19aae86-83f1-406a-9cbf-7b46b11fcb00-kube-api-access-75cs6\") pod \"maas-controller-5dfc4f849f-pjjzp\" (UID: \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\") " pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:37.043892 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.043866    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:37.132013 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.131960    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:37.221332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.221308    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:37.330012 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.329987    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8486858879-ttlr5"]
Oct 30 18:44:37.332767 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.332751    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:37.345705 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.345683    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8486858879-ttlr5"]
Oct 30 18:44:37.407158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.407081    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5dfc4f849f-pjjzp"]
Oct 30 18:44:37.410376 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:44:37.410350    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode19aae86_83f1_406a_9cbf_7b46b11fcb00.slice/crio-305cc9ac1f1920e291e5a60c677f285555212a0ab36338e3d357e586dd62639b WatchSource:0}: Error finding container 305cc9ac1f1920e291e5a60c677f285555212a0ab36338e3d357e586dd62639b: Status 404 returned error can't find the container with id 305cc9ac1f1920e291e5a60c677f285555212a0ab36338e3d357e586dd62639b
Oct 30 18:44:37.412403 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.412382    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9-webhook-cert\") pod \"maas-controller-8486858879-ttlr5\" (UID: \"e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9\") " pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:37.412552 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.412417    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9-metrics-certs\") pod \"maas-controller-8486858879-ttlr5\" (UID: \"e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9\") " pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:37.412552 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.412446    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5kxd\" (UniqueName: \"kubernetes.io/projected/e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9-kube-api-access-j5kxd\") pod \"maas-controller-8486858879-ttlr5\" (UID: \"e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9\") " pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:37.467668 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.467640    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dfc4f849f-pjjzp" event={"ID":"e19aae86-83f1-406a-9cbf-7b46b11fcb00","Type":"ContainerStarted","Data":"305cc9ac1f1920e291e5a60c677f285555212a0ab36338e3d357e586dd62639b"}
Oct 30 18:44:37.512997 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.512969    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9-webhook-cert\") pod \"maas-controller-8486858879-ttlr5\" (UID: \"e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9\") " pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:37.513138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.513014    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9-metrics-certs\") pod \"maas-controller-8486858879-ttlr5\" (UID: \"e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9\") " pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:37.513138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.513041    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5kxd\" (UniqueName: \"kubernetes.io/projected/e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9-kube-api-access-j5kxd\") pod \"maas-controller-8486858879-ttlr5\" (UID: \"e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9\") " pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:37.515665 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.515640    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9-metrics-certs\") pod \"maas-controller-8486858879-ttlr5\" (UID: \"e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9\") " pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:37.515743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.515640    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9-webhook-cert\") pod \"maas-controller-8486858879-ttlr5\" (UID: \"e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9\") " pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:37.521342 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.521324    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5kxd\" (UniqueName: \"kubernetes.io/projected/e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9-kube-api-access-j5kxd\") pod \"maas-controller-8486858879-ttlr5\" (UID: \"e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9\") " pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:37.644518 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.644486    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:37.915121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:37.915072    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8486858879-ttlr5"]
Oct 30 18:44:37.917542 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:44:37.917517    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1befbbd_48f4_49d2_9ecf_cb6c95d15ab9.slice/crio-d74cafa62f401f5136ceaf77c99bd1a8c022ee08f58c9a28a07dee1af7274985 WatchSource:0}: Error finding container d74cafa62f401f5136ceaf77c99bd1a8c022ee08f58c9a28a07dee1af7274985: Status 404 returned error can't find the container with id d74cafa62f401f5136ceaf77c99bd1a8c022ee08f58c9a28a07dee1af7274985
Oct 30 18:44:38.473238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:38.473203    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8486858879-ttlr5" event={"ID":"e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9","Type":"ContainerStarted","Data":"d74cafa62f401f5136ceaf77c99bd1a8c022ee08f58c9a28a07dee1af7274985"}
Oct 30 18:44:41.489295 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:41.489253    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8486858879-ttlr5" event={"ID":"e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9","Type":"ContainerStarted","Data":"82f709fd7f3bec637ac0d83da5454ba0ed592d4f5d1039bbb93a0a44485dc7a1"}
Oct 30 18:44:41.489733 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:41.489422    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:41.490719 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:41.490698    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dfc4f849f-pjjzp" event={"ID":"e19aae86-83f1-406a-9cbf-7b46b11fcb00","Type":"ContainerStarted","Data":"cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22"}
Oct 30 18:44:41.490836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:41.490824    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:41.578535 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:41.578441    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5dfc4f849f-pjjzp" podStartSLOduration=1.707854002 podStartE2EDuration="5.578429063s" totalImagesPullingTime="3.870575061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:41.577395872 +0000 UTC m=+503.293740949" watchObservedRunningTime="2026-10-30 18:44:41.578429063 +0000 UTC m=+503.294774159"
Oct 30 18:44:41.579319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:41.579290    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-8486858879-ttlr5" podStartSLOduration=1.207594273 podStartE2EDuration="4.579282467s" totalImagesPullingTime="3.371688194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:41.52441445 +0000 UTC m=+503.240759516" watchObservedRunningTime="2026-10-30 18:44:41.579282467 +0000 UTC m=+503.295627543"
Oct 30 18:44:41.646608 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:41.646578    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:41.842773 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:41.842688    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:43.103843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.103811    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"]
Oct 30 18:44:43.107334 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.107310    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.113145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.113011    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-gq8q5\""
Oct 30 18:44:43.113145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.113085    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 18:44:43.115987 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.115967    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 18:44:43.116112 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.116008    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 18:44:43.121359 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.121336    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"]
Oct 30 18:44:43.259080 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.259049    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:43.262154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.262121    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnbtb\" (UniqueName: \"kubernetes.io/projected/7242bc65-404e-481b-b596-968eb61f0f1c-kube-api-access-lnbtb\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-gx2x5\" (UID: \"7242bc65-404e-481b-b596-968eb61f0f1c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.262281 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.262182    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7242bc65-404e-481b-b596-968eb61f0f1c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-gx2x5\" (UID: \"7242bc65-404e-481b-b596-968eb61f0f1c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.262281 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.262213    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7242bc65-404e-481b-b596-968eb61f0f1c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-gx2x5\" (UID: \"7242bc65-404e-481b-b596-968eb61f0f1c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.262363 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.262287    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7242bc65-404e-481b-b596-968eb61f0f1c-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-gx2x5\" (UID: \"7242bc65-404e-481b-b596-968eb61f0f1c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.302456 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:44:43.302421    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:44:43.363112 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.363027    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnbtb\" (UniqueName: \"kubernetes.io/projected/7242bc65-404e-481b-b596-968eb61f0f1c-kube-api-access-lnbtb\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-gx2x5\" (UID: \"7242bc65-404e-481b-b596-968eb61f0f1c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.363112 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.363065    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7242bc65-404e-481b-b596-968eb61f0f1c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-gx2x5\" (UID: \"7242bc65-404e-481b-b596-968eb61f0f1c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.363112 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.363089    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7242bc65-404e-481b-b596-968eb61f0f1c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-gx2x5\" (UID: \"7242bc65-404e-481b-b596-968eb61f0f1c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.363366 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.363246    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7242bc65-404e-481b-b596-968eb61f0f1c-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-gx2x5\" (UID: \"7242bc65-404e-481b-b596-968eb61f0f1c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.363825 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.363801    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7242bc65-404e-481b-b596-968eb61f0f1c-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-gx2x5\" (UID: \"7242bc65-404e-481b-b596-968eb61f0f1c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.364041 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.364023    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7242bc65-404e-481b-b596-968eb61f0f1c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-gx2x5\" (UID: \"7242bc65-404e-481b-b596-968eb61f0f1c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.365554 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.365537    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7242bc65-404e-481b-b596-968eb61f0f1c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-gx2x5\" (UID: \"7242bc65-404e-481b-b596-968eb61f0f1c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.371135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.371087    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnbtb\" (UniqueName: \"kubernetes.io/projected/7242bc65-404e-481b-b596-968eb61f0f1c-kube-api-access-lnbtb\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-gx2x5\" (UID: \"7242bc65-404e-481b-b596-968eb61f0f1c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.423431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.423410    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:43.442270 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.442247    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:43.706183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:43.706149    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"]
Oct 30 18:44:43.709065 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:44:43.709040    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7242bc65_404e_481b_b596_968eb61f0f1c.slice/crio-2e7e14db30c96fd2aba784fc95db63e223aab58d91b39ad0151092ca2ff4ff53 WatchSource:0}: Error finding container 2e7e14db30c96fd2aba784fc95db63e223aab58d91b39ad0151092ca2ff4ff53: Status 404 returned error can't find the container with id 2e7e14db30c96fd2aba784fc95db63e223aab58d91b39ad0151092ca2ff4ff53
Oct 30 18:44:44.503967 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:44.503925    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5" event={"ID":"7242bc65-404e-481b-b596-968eb61f0f1c","Type":"ContainerStarted","Data":"2e7e14db30c96fd2aba784fc95db63e223aab58d91b39ad0151092ca2ff4ff53"}
Oct 30 18:44:52.497216 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:52.497186    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:52.497669 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:52.497269    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8486858879-ttlr5"
Oct 30 18:44:52.565003 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:52.564972    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5dfc4f849f-pjjzp"]
Oct 30 18:44:52.565556 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:52.565515    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-5dfc4f849f-pjjzp" podUID="e19aae86-83f1-406a-9cbf-7b46b11fcb00" containerName="manager" containerID="cri-o://cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22" gracePeriod=30
Oct 30 18:44:52.842861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:52.842836    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:52.943025 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:52.942990    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-metrics-certs\") pod \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\" (UID: \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\") "
Oct 30 18:44:52.943213 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:52.943043    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e19aae86-83f1-406a-9cbf-7b46b11fcb00-kube-api-access-75cs6\" (UniqueName: \"kubernetes.io/projected/e19aae86-83f1-406a-9cbf-7b46b11fcb00-kube-api-access-75cs6\") pod \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\" (UID: \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\") "
Oct 30 18:44:52.943294 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:52.943207    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-webhook-cert\") pod \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\" (UID: \"e19aae86-83f1-406a-9cbf-7b46b11fcb00\") "
Oct 30 18:44:52.945573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:52.945544    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-metrics-certs" pod "e19aae86-83f1-406a-9cbf-7b46b11fcb00" (UID: "e19aae86-83f1-406a-9cbf-7b46b11fcb00"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:44:52.945683 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:52.945578    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e19aae86-83f1-406a-9cbf-7b46b11fcb00-kube-api-access-75cs6" pod "e19aae86-83f1-406a-9cbf-7b46b11fcb00" (UID: "e19aae86-83f1-406a-9cbf-7b46b11fcb00"). InnerVolumeSpecName "kube-api-access-75cs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:44:52.945763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:52.945734    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-webhook-cert" pod "e19aae86-83f1-406a-9cbf-7b46b11fcb00" (UID: "e19aae86-83f1-406a-9cbf-7b46b11fcb00"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:44:53.044341 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:53.044310    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75cs6\" (UniqueName: \"kubernetes.io/projected/e19aae86-83f1-406a-9cbf-7b46b11fcb00-kube-api-access-75cs6\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:53.044341 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:53.044344    4925 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-webhook-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:53.044512 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:53.044361    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e19aae86-83f1-406a-9cbf-7b46b11fcb00-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:44:53.549376 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:53.549342    4925 generic.go:356] "Generic (PLEG): container finished" podID="e19aae86-83f1-406a-9cbf-7b46b11fcb00" containerID="cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22" exitCode=0
Oct 30 18:44:53.549860 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:53.549404    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dfc4f849f-pjjzp"
Oct 30 18:44:53.549860 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:53.549449    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dfc4f849f-pjjzp" event={"ID":"e19aae86-83f1-406a-9cbf-7b46b11fcb00","Type":"ContainerDied","Data":"cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22"}
Oct 30 18:44:53.549860 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:53.549517    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dfc4f849f-pjjzp" event={"ID":"e19aae86-83f1-406a-9cbf-7b46b11fcb00","Type":"ContainerDied","Data":"305cc9ac1f1920e291e5a60c677f285555212a0ab36338e3d357e586dd62639b"}
Oct 30 18:44:53.549860 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:53.549552    4925 scope.go:122] "RemoveContainer" containerID="cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22"
Oct 30 18:44:53.559510 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:53.559490    4925 scope.go:122] "RemoveContainer" containerID="cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22"
Oct 30 18:44:53.559823 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:44:53.559799    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22\": container with ID starting with cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22 not found: ID does not exist" containerID="cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22"
Oct 30 18:44:53.559934 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:53.559830    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22"} err="failed to get container status \"cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22\": rpc error: code = NotFound desc = could not find container \"cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22\": container with ID starting with cb74a454a127a14d3a3015fa08d46d411290e0bbf8a0797e610b0fc3d4578a22 not found: ID does not exist"
Oct 30 18:44:53.574085 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:53.574065    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5dfc4f849f-pjjzp"]
Oct 30 18:44:53.576477 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:53.576456    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5dfc4f849f-pjjzp"]
Oct 30 18:44:54.420283 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:54.420252    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e19aae86-83f1-406a-9cbf-7b46b11fcb00" path="/var/lib/kubelet/pods/e19aae86-83f1-406a-9cbf-7b46b11fcb00/volumes"
Oct 30 18:44:55.561239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:55.561154    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5" event={"ID":"7242bc65-404e-481b-b596-968eb61f0f1c","Type":"ContainerStarted","Data":"c5e4df5a59e70cb4beb0f8c66f5eb9b5c4952bd105bdf03f06281a755b72ac66"}
Oct 30 18:44:55.561639 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:55.561300    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:44:55.582490 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:44:55.582440    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5" podStartSLOduration=1.084331543 podStartE2EDuration="12.582428081s" totalImagesPullingTime="11.498096538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:55.581301142 +0000 UTC m=+517.297646213" watchObservedRunningTime="2026-10-30 18:44:55.582428081 +0000 UTC m=+517.298773169"
Oct 30 18:45:06.571404 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:06.571371    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-gx2x5"
Oct 30 18:45:08.311596 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:08.311560    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:11.858931 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:11.858891    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:12.121645 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:12.121557    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:13.192431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:13.192398    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:13.302212 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:45:13.302177    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:45:13.389648 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:13.389611    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:24.597647 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:24.597613    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:24.776916 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:24.776881    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:25.052905 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.052865    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"]
Oct 30 18:45:25.053306 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.053287    4925 cpu_manager.go:414] "containerMap: removing container" podUID="e19aae86-83f1-406a-9cbf-7b46b11fcb00" containerName="manager"
Oct 30 18:45:25.053408 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.053307    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e19aae86-83f1-406a-9cbf-7b46b11fcb00" containerName="manager"
Oct 30 18:45:25.053465 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.053431    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="e19aae86-83f1-406a-9cbf-7b46b11fcb00" containerName="manager"
Oct 30 18:45:25.331529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.331453    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"]
Oct 30 18:45:25.331664 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.331536    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.340067 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.340048    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 18:45:25.340958 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.340943    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 18:45:25.341024 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.340975    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pd8db\""
Oct 30 18:45:25.343091 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.343072    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 18:45:25.343225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.343213    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 18:45:25.396888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.396863    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c8e0d4-23c8-407b-b624-c84b8f2f31c6-openshift-service-ca\") pod \"maas-api-76ff54bc44-n2nzz\" (UID: \"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6\") " pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.396991 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.396895    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r99gx\" (UniqueName: \"kubernetes.io/projected/a7c8e0d4-23c8-407b-b624-c84b8f2f31c6-kube-api-access-r99gx\") pod \"maas-api-76ff54bc44-n2nzz\" (UID: \"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6\") " pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.396991 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.396932    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c8e0d4-23c8-407b-b624-c84b8f2f31c6-maas-api-tls\") pod \"maas-api-76ff54bc44-n2nzz\" (UID: \"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6\") " pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.397142 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.397120    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7c8e0d4-23c8-407b-b624-c84b8f2f31c6-metrics-certs\") pod \"maas-api-76ff54bc44-n2nzz\" (UID: \"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6\") " pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.498433 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.498404    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c8e0d4-23c8-407b-b624-c84b8f2f31c6-openshift-service-ca\") pod \"maas-api-76ff54bc44-n2nzz\" (UID: \"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6\") " pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.498536 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.498445    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r99gx\" (UniqueName: \"kubernetes.io/projected/a7c8e0d4-23c8-407b-b624-c84b8f2f31c6-kube-api-access-r99gx\") pod \"maas-api-76ff54bc44-n2nzz\" (UID: \"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6\") " pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.498536 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.498509    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c8e0d4-23c8-407b-b624-c84b8f2f31c6-maas-api-tls\") pod \"maas-api-76ff54bc44-n2nzz\" (UID: \"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6\") " pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.498646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.498552    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7c8e0d4-23c8-407b-b624-c84b8f2f31c6-metrics-certs\") pod \"maas-api-76ff54bc44-n2nzz\" (UID: \"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6\") " pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.499200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.499180    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c8e0d4-23c8-407b-b624-c84b8f2f31c6-openshift-service-ca\") pod \"maas-api-76ff54bc44-n2nzz\" (UID: \"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6\") " pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.501188 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.501168    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c8e0d4-23c8-407b-b624-c84b8f2f31c6-maas-api-tls\") pod \"maas-api-76ff54bc44-n2nzz\" (UID: \"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6\") " pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.501265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.501246    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7c8e0d4-23c8-407b-b624-c84b8f2f31c6-metrics-certs\") pod \"maas-api-76ff54bc44-n2nzz\" (UID: \"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6\") " pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.508261 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.508240    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r99gx\" (UniqueName: \"kubernetes.io/projected/a7c8e0d4-23c8-407b-b624-c84b8f2f31c6-kube-api-access-r99gx\") pod \"maas-api-76ff54bc44-n2nzz\" (UID: \"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6\") " pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.659887 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.659864    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:25.949225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:25.949141    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"]
Oct 30 18:45:25.952529 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:45:25.952497    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7c8e0d4_23c8_407b_b624_c84b8f2f31c6.slice/crio-c4208c1dcbc5c67c12fd7e30face16e0c4c3327d635f1674d1878592d07fdd7e WatchSource:0}: Error finding container c4208c1dcbc5c67c12fd7e30face16e0c4c3327d635f1674d1878592d07fdd7e: Status 404 returned error can't find the container with id c4208c1dcbc5c67c12fd7e30face16e0c4c3327d635f1674d1878592d07fdd7e
Oct 30 18:45:26.697916 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:26.697883    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz" event={"ID":"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6","Type":"ContainerStarted","Data":"c4208c1dcbc5c67c12fd7e30face16e0c4c3327d635f1674d1878592d07fdd7e"}
Oct 30 18:45:28.707046 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:28.707009    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz" event={"ID":"a7c8e0d4-23c8-407b-b624-c84b8f2f31c6","Type":"ContainerStarted","Data":"99bc499eb5d7c82a2a7747d569e12b1bb4bf8ac3fc2e8e226bf62590e98bcc89"}
Oct 30 18:45:28.707443 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:28.707072    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:28.726206 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:28.726161    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz" podStartSLOduration=1.8135951179999998 podStartE2EDuration="3.726146086s" totalImagesPullingTime="1.912550968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:28.725614311 +0000 UTC m=+550.441959398" watchObservedRunningTime="2026-10-30 18:45:28.726146086 +0000 UTC m=+550.442491162"
Oct 30 18:45:34.717937 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:45:34.717905    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-76ff54bc44-n2nzz"
Oct 30 18:45:43.305652 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:45:43.302832    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:46:02.343649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.343565    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-nmwkt"]
Oct 30 18:46:02.348934 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.348904    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt"
Oct 30 18:46:02.379375 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.379319    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 18:46:02.379532 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.379421    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 18:46:02.379532 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.379494    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-2rfhm\""
Oct 30 18:46:02.380389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.380368    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-nmwkt"]
Oct 30 18:46:02.452955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.452925    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:02.473901 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.473874    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9e9a00-3685-421b-99c7-60bb7896abe5-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-nmwkt\" (UID: \"fa9e9a00-3685-421b-99c7-60bb7896abe5\") " pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt"
Oct 30 18:46:02.474002 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.473902    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27r5x\" (UniqueName: \"kubernetes.io/projected/fa9e9a00-3685-421b-99c7-60bb7896abe5-kube-api-access-27r5x\") pod \"ai-gateway-controller-7d85675f98-nmwkt\" (UID: \"fa9e9a00-3685-421b-99c7-60bb7896abe5\") " pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt"
Oct 30 18:46:02.574906 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.574878    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9e9a00-3685-421b-99c7-60bb7896abe5-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-nmwkt\" (UID: \"fa9e9a00-3685-421b-99c7-60bb7896abe5\") " pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt"
Oct 30 18:46:02.575041 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.574910    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27r5x\" (UniqueName: \"kubernetes.io/projected/fa9e9a00-3685-421b-99c7-60bb7896abe5-kube-api-access-27r5x\") pod \"ai-gateway-controller-7d85675f98-nmwkt\" (UID: \"fa9e9a00-3685-421b-99c7-60bb7896abe5\") " pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt"
Oct 30 18:46:02.577577 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.577554    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9e9a00-3685-421b-99c7-60bb7896abe5-metrics-certs\") pod \"ai-gateway-controller-7d85675f98-nmwkt\" (UID: \"fa9e9a00-3685-421b-99c7-60bb7896abe5\") " pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt"
Oct 30 18:46:02.589619 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.589592    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27r5x\" (UniqueName: \"kubernetes.io/projected/fa9e9a00-3685-421b-99c7-60bb7896abe5-kube-api-access-27r5x\") pod \"ai-gateway-controller-7d85675f98-nmwkt\" (UID: \"fa9e9a00-3685-421b-99c7-60bb7896abe5\") " pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt"
Oct 30 18:46:02.663331 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.663309    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt"
Oct 30 18:46:02.669563 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.669531    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:02.937218 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:02.937192    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-nmwkt"]
Oct 30 18:46:02.940238 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:46:02.940211    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9e9a00_3685_421b_99c7_60bb7896abe5.slice/crio-c5cd79f9cf1564af07f9606272f89fc1aff599bedc5aa83bd305d78108d3f8da WatchSource:0}: Error finding container c5cd79f9cf1564af07f9606272f89fc1aff599bedc5aa83bd305d78108d3f8da: Status 404 returned error can't find the container with id c5cd79f9cf1564af07f9606272f89fc1aff599bedc5aa83bd305d78108d3f8da
Oct 30 18:46:03.870016 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:03.869979    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt" event={"ID":"fa9e9a00-3685-421b-99c7-60bb7896abe5","Type":"ContainerStarted","Data":"c5cd79f9cf1564af07f9606272f89fc1aff599bedc5aa83bd305d78108d3f8da"}
Oct 30 18:46:04.878058 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:04.877963    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt" event={"ID":"fa9e9a00-3685-421b-99c7-60bb7896abe5","Type":"ContainerStarted","Data":"6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9"}
Oct 30 18:46:04.878521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:04.878072    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt"
Oct 30 18:46:04.909961 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:04.909911    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt" podStartSLOduration=1.410945103 podStartE2EDuration="2.909896064s" totalImagesPullingTime="1.498950961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:04.907300875 +0000 UTC m=+586.623645951" watchObservedRunningTime="2026-10-30 18:46:04.909896064 +0000 UTC m=+586.626241139"
Oct 30 18:46:05.432858 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.432827    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"]
Oct 30 18:46:05.436315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.436295    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"
Oct 30 18:46:05.445842 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.445818    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Oct 30 18:46:05.447527 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.447504    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"]
Oct 30 18:46:05.469188 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.469167    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rs5bm\""
Oct 30 18:46:05.483939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.483917    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-vfsrt"]
Oct 30 18:46:05.487203 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.487187    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt"
Oct 30 18:46:05.509852 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.509830    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-vfsrt"]
Oct 30 18:46:05.518169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.518145    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:05.601049 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.601013    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77704542-8e8e-4d3e-a4bd-e07d4f0e79a0-plugins-config-volume\") pod \"payload-processing-5f78687d5b-vfsrt\" (UID: \"77704542-8e8e-4d3e-a4bd-e07d4f0e79a0\") " pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt"
Oct 30 18:46:05.601049 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.601047    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6edaba68-471d-475d-9a07-16ecd7bd4e8e-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-6hs9t\" (UID: \"6edaba68-471d-475d-9a07-16ecd7bd4e8e\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"
Oct 30 18:46:05.601280 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.601146    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jqvj\" (UniqueName: \"kubernetes.io/projected/6edaba68-471d-475d-9a07-16ecd7bd4e8e-kube-api-access-2jqvj\") pod \"payload-pre-processing-7b98bb678c-6hs9t\" (UID: \"6edaba68-471d-475d-9a07-16ecd7bd4e8e\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"
Oct 30 18:46:05.601280 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.601175    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q687\" (UniqueName: \"kubernetes.io/projected/77704542-8e8e-4d3e-a4bd-e07d4f0e79a0-kube-api-access-5q687\") pod \"payload-processing-5f78687d5b-vfsrt\" (UID: \"77704542-8e8e-4d3e-a4bd-e07d4f0e79a0\") " pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt"
Oct 30 18:46:05.702028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.701950    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77704542-8e8e-4d3e-a4bd-e07d4f0e79a0-plugins-config-volume\") pod \"payload-processing-5f78687d5b-vfsrt\" (UID: \"77704542-8e8e-4d3e-a4bd-e07d4f0e79a0\") " pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt"
Oct 30 18:46:05.702028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.702007    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6edaba68-471d-475d-9a07-16ecd7bd4e8e-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-6hs9t\" (UID: \"6edaba68-471d-475d-9a07-16ecd7bd4e8e\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"
Oct 30 18:46:05.702265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.702051    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jqvj\" (UniqueName: \"kubernetes.io/projected/6edaba68-471d-475d-9a07-16ecd7bd4e8e-kube-api-access-2jqvj\") pod \"payload-pre-processing-7b98bb678c-6hs9t\" (UID: \"6edaba68-471d-475d-9a07-16ecd7bd4e8e\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"
Oct 30 18:46:05.702265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.702115    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q687\" (UniqueName: \"kubernetes.io/projected/77704542-8e8e-4d3e-a4bd-e07d4f0e79a0-kube-api-access-5q687\") pod \"payload-processing-5f78687d5b-vfsrt\" (UID: \"77704542-8e8e-4d3e-a4bd-e07d4f0e79a0\") " pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt"
Oct 30 18:46:05.702742 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.702716    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/77704542-8e8e-4d3e-a4bd-e07d4f0e79a0-plugins-config-volume\") pod \"payload-processing-5f78687d5b-vfsrt\" (UID: \"77704542-8e8e-4d3e-a4bd-e07d4f0e79a0\") " pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt"
Oct 30 18:46:05.702848 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.702830    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6edaba68-471d-475d-9a07-16ecd7bd4e8e-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-6hs9t\" (UID: \"6edaba68-471d-475d-9a07-16ecd7bd4e8e\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"
Oct 30 18:46:05.715821 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.715786    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q687\" (UniqueName: \"kubernetes.io/projected/77704542-8e8e-4d3e-a4bd-e07d4f0e79a0-kube-api-access-5q687\") pod \"payload-processing-5f78687d5b-vfsrt\" (UID: \"77704542-8e8e-4d3e-a4bd-e07d4f0e79a0\") " pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt"
Oct 30 18:46:05.716736 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.716715    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jqvj\" (UniqueName: \"kubernetes.io/projected/6edaba68-471d-475d-9a07-16ecd7bd4e8e-kube-api-access-2jqvj\") pod \"payload-pre-processing-7b98bb678c-6hs9t\" (UID: \"6edaba68-471d-475d-9a07-16ecd7bd4e8e\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"
Oct 30 18:46:05.749702 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.749678    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"
Oct 30 18:46:05.781064 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.781018    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:05.799495 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:05.799254    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt"
Oct 30 18:46:06.032340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:06.032278    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"]
Oct 30 18:46:06.035154 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:46:06.035132    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6edaba68_471d_475d_9a07_16ecd7bd4e8e.slice/crio-8281661e05b8e2d0018c6a98e4b4821496179fc88d6b460887f793580cb68613 WatchSource:0}: Error finding container 8281661e05b8e2d0018c6a98e4b4821496179fc88d6b460887f793580cb68613: Status 404 returned error can't find the container with id 8281661e05b8e2d0018c6a98e4b4821496179fc88d6b460887f793580cb68613
Oct 30 18:46:06.112341 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:06.112322    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-vfsrt"]
Oct 30 18:46:06.114606 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:46:06.114575    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77704542_8e8e_4d3e_a4bd_e07d4f0e79a0.slice/crio-87f1aa77a2c90ad865139089c89aba7e7e5b7beb0ca3299bb1feb2c70fe759c7 WatchSource:0}: Error finding container 87f1aa77a2c90ad865139089c89aba7e7e5b7beb0ca3299bb1feb2c70fe759c7: Status 404 returned error can't find the container with id 87f1aa77a2c90ad865139089c89aba7e7e5b7beb0ca3299bb1feb2c70fe759c7
Oct 30 18:46:06.891536 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:06.891208    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t" event={"ID":"6edaba68-471d-475d-9a07-16ecd7bd4e8e","Type":"ContainerStarted","Data":"8281661e05b8e2d0018c6a98e4b4821496179fc88d6b460887f793580cb68613"}
Oct 30 18:46:06.892820 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:06.892785    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt" event={"ID":"77704542-8e8e-4d3e-a4bd-e07d4f0e79a0","Type":"ContainerStarted","Data":"87f1aa77a2c90ad865139089c89aba7e7e5b7beb0ca3299bb1feb2c70fe759c7"}
Oct 30 18:46:08.315831 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:08.315801    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:08.903599 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:08.903568    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t" event={"ID":"6edaba68-471d-475d-9a07-16ecd7bd4e8e","Type":"ContainerStarted","Data":"0924f580cab091847dca8e6bfbb03af4134b3c45bf53ae5d5854df40dce70902"}
Oct 30 18:46:08.903799 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:08.903683    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"
Oct 30 18:46:08.905212 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:08.905191    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt" event={"ID":"77704542-8e8e-4d3e-a4bd-e07d4f0e79a0","Type":"ContainerStarted","Data":"8b4c3cac5942e92ff5f3e59a6efff20ae4ca42654fc431b2dd081d3ec21d7e1d"}
Oct 30 18:46:08.905317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:08.905236    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt"
Oct 30 18:46:08.926757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:08.926711    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t" podStartSLOduration=1.684507755 podStartE2EDuration="3.926697491s" totalImagesPullingTime="2.242189736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:08.924310883 +0000 UTC m=+590.640655969" watchObservedRunningTime="2026-10-30 18:46:08.926697491 +0000 UTC m=+590.643042568"
Oct 30 18:46:08.941803 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:08.941766    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt" podStartSLOduration=1.718584943 podStartE2EDuration="3.941754976s" totalImagesPullingTime="2.223170033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:08.94093374 +0000 UTC m=+590.657278817" watchObservedRunningTime="2026-10-30 18:46:08.941754976 +0000 UTC m=+590.658100051"
Oct 30 18:46:13.300416 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:46:13.300381    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:46:15.885529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:15.885494    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt"
Oct 30 18:46:16.141729 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:16.141656    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"]
Oct 30 18:46:16.145457 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:16.145441    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"
Oct 30 18:46:16.155050 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:16.155026    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"]
Oct 30 18:46:16.186851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:16.186829    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c536d86d-2397-47bf-a44f-47b0bf40dd02-metrics-certs\") pod \"ai-gateway-controller-6c45775ff9-c2ghb\" (UID: \"c536d86d-2397-47bf-a44f-47b0bf40dd02\") " pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"
Oct 30 18:46:16.186985 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:16.186890    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jblgz\" (UniqueName: \"kubernetes.io/projected/c536d86d-2397-47bf-a44f-47b0bf40dd02-kube-api-access-jblgz\") pod \"ai-gateway-controller-6c45775ff9-c2ghb\" (UID: \"c536d86d-2397-47bf-a44f-47b0bf40dd02\") " pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"
Oct 30 18:46:16.287388 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:16.287358    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jblgz\" (UniqueName: \"kubernetes.io/projected/c536d86d-2397-47bf-a44f-47b0bf40dd02-kube-api-access-jblgz\") pod \"ai-gateway-controller-6c45775ff9-c2ghb\" (UID: \"c536d86d-2397-47bf-a44f-47b0bf40dd02\") " pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"
Oct 30 18:46:16.287521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:16.287398    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c536d86d-2397-47bf-a44f-47b0bf40dd02-metrics-certs\") pod \"ai-gateway-controller-6c45775ff9-c2ghb\" (UID: \"c536d86d-2397-47bf-a44f-47b0bf40dd02\") " pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"
Oct 30 18:46:16.289962 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:16.289941    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c536d86d-2397-47bf-a44f-47b0bf40dd02-metrics-certs\") pod \"ai-gateway-controller-6c45775ff9-c2ghb\" (UID: \"c536d86d-2397-47bf-a44f-47b0bf40dd02\") " pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"
Oct 30 18:46:16.296480 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:16.296453    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jblgz\" (UniqueName: \"kubernetes.io/projected/c536d86d-2397-47bf-a44f-47b0bf40dd02-kube-api-access-jblgz\") pod \"ai-gateway-controller-6c45775ff9-c2ghb\" (UID: \"c536d86d-2397-47bf-a44f-47b0bf40dd02\") " pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"
Oct 30 18:46:16.458889 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:16.458865    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"
Oct 30 18:46:16.730836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:16.730710    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"]
Oct 30 18:46:16.733594 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:46:16.733566    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc536d86d_2397_47bf_a44f_47b0bf40dd02.slice/crio-deef36c370383a7ba8ef975833d3b3fdb5673145b86c71a679ef9280638cf5f1 WatchSource:0}: Error finding container deef36c370383a7ba8ef975833d3b3fdb5673145b86c71a679ef9280638cf5f1: Status 404 returned error can't find the container with id deef36c370383a7ba8ef975833d3b3fdb5673145b86c71a679ef9280638cf5f1
Oct 30 18:46:16.949446 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:16.949412    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb" event={"ID":"c536d86d-2397-47bf-a44f-47b0bf40dd02","Type":"ContainerStarted","Data":"deef36c370383a7ba8ef975833d3b3fdb5673145b86c71a679ef9280638cf5f1"}
Oct 30 18:46:17.957042 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:17.956999    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb" event={"ID":"c536d86d-2397-47bf-a44f-47b0bf40dd02","Type":"ContainerStarted","Data":"10d8614c7c8e1c64b879b625346ba72528a0aa717a516335694d947278600075"}
Oct 30 18:46:17.957502 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:17.957127    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"
Oct 30 18:46:17.977375 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:17.977326    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb" podStartSLOduration=1.506176885 podStartE2EDuration="1.977310628s" totalImagesPullingTime="471.133743ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:17.975447291 +0000 UTC m=+599.691792369" watchObservedRunningTime="2026-10-30 18:46:17.977310628 +0000 UTC m=+599.693655705"
Oct 30 18:46:18.460415 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:18.460392    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 18:46:18.460679 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:18.460653    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 18:46:19.913571 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:19.913536    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5f78687d5b-vfsrt"
Oct 30 18:46:19.914015 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:19.913610    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-6hs9t"
Oct 30 18:46:20.156425 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:20.156395    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:20.363108 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:20.362996    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:28.964212 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:28.964180    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-6c45775ff9-c2ghb"
Oct 30 18:46:29.006920 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:29.006892    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-nmwkt"]
Oct 30 18:46:29.007132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:29.007087    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt" podUID="fa9e9a00-3685-421b-99c7-60bb7896abe5" containerName="manager" containerID="cri-o://6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9" gracePeriod=30
Oct 30 18:46:29.251895 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:29.251873    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt"
Oct 30 18:46:29.285474 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:29.285452    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa9e9a00-3685-421b-99c7-60bb7896abe5-kube-api-access-27r5x\" (UniqueName: \"kubernetes.io/projected/fa9e9a00-3685-421b-99c7-60bb7896abe5-kube-api-access-27r5x\") pod \"fa9e9a00-3685-421b-99c7-60bb7896abe5\" (UID: \"fa9e9a00-3685-421b-99c7-60bb7896abe5\") "
Oct 30 18:46:29.285600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:29.285566    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa9e9a00-3685-421b-99c7-60bb7896abe5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9e9a00-3685-421b-99c7-60bb7896abe5-metrics-certs\") pod \"fa9e9a00-3685-421b-99c7-60bb7896abe5\" (UID: \"fa9e9a00-3685-421b-99c7-60bb7896abe5\") "
Oct 30 18:46:29.287799 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:29.287775    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa9e9a00-3685-421b-99c7-60bb7896abe5-kube-api-access-27r5x" pod "fa9e9a00-3685-421b-99c7-60bb7896abe5" (UID: "fa9e9a00-3685-421b-99c7-60bb7896abe5"). InnerVolumeSpecName "kube-api-access-27r5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:29.287894 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:29.287779    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa9e9a00-3685-421b-99c7-60bb7896abe5-metrics-certs" pod "fa9e9a00-3685-421b-99c7-60bb7896abe5" (UID: "fa9e9a00-3685-421b-99c7-60bb7896abe5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:29.386158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:29.386134    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa9e9a00-3685-421b-99c7-60bb7896abe5-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:46:29.386158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:29.386155    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27r5x\" (UniqueName: \"kubernetes.io/projected/fa9e9a00-3685-421b-99c7-60bb7896abe5-kube-api-access-27r5x\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:46:30.014409 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:30.014374    4925 generic.go:356] "Generic (PLEG): container finished" podID="fa9e9a00-3685-421b-99c7-60bb7896abe5" containerID="6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9" exitCode=0
Oct 30 18:46:30.014796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:30.014424    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt"
Oct 30 18:46:30.014796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:30.014424    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt" event={"ID":"fa9e9a00-3685-421b-99c7-60bb7896abe5","Type":"ContainerDied","Data":"6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9"}
Oct 30 18:46:30.014796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:30.014480    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d85675f98-nmwkt" event={"ID":"fa9e9a00-3685-421b-99c7-60bb7896abe5","Type":"ContainerDied","Data":"c5cd79f9cf1564af07f9606272f89fc1aff599bedc5aa83bd305d78108d3f8da"}
Oct 30 18:46:30.014796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:30.014511    4925 scope.go:122] "RemoveContainer" containerID="6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9"
Oct 30 18:46:30.024779 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:30.024760    4925 scope.go:122] "RemoveContainer" containerID="6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9"
Oct 30 18:46:30.025070 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:46:30.025049    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9\": container with ID starting with 6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9 not found: ID does not exist" containerID="6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9"
Oct 30 18:46:30.025136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:30.025078    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9"} err="failed to get container status \"6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9\": rpc error: code = NotFound desc = could not find container \"6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9\": container with ID starting with 6f943bd52e8be121d56448bbc9899dd68bc34a1ce82be09a4326f5de94c0d2b9 not found: ID does not exist"
Oct 30 18:46:30.036613 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:30.036588    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-nmwkt"]
Oct 30 18:46:30.040254 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:30.040218    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-7d85675f98-nmwkt"]
Oct 30 18:46:30.419365 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:30.419337    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa9e9a00-3685-421b-99c7-60bb7896abe5" path="/var/lib/kubelet/pods/fa9e9a00-3685-421b-99c7-60bb7896abe5/volumes"
Oct 30 18:46:30.727355 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:30.727277    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:31.242386 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:31.242348    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:32.490521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:32.490492    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:32.679083 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:32.679053    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:35.339184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.339149    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"]
Oct 30 18:46:35.339628 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.339611    4925 cpu_manager.go:414] "containerMap: removing container" podUID="fa9e9a00-3685-421b-99c7-60bb7896abe5" containerName="manager"
Oct 30 18:46:35.339671 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.339631    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa9e9a00-3685-421b-99c7-60bb7896abe5" containerName="manager"
Oct 30 18:46:35.339719 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.339710    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa9e9a00-3685-421b-99c7-60bb7896abe5" containerName="manager"
Oct 30 18:46:35.342936 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.342916    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.351803 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.351747    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 18:46:35.352201 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.352170    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 18:46:35.352310 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.352255    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 18:46:35.352310 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.352273    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nz76v\""
Oct 30 18:46:35.354789 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.354763    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"]
Oct 30 18:46:35.428909 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.428878    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d32df70-08f9-494a-81d5-868f1511dcb4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.429023 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.428936    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d32df70-08f9-494a-81d5-868f1511dcb4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.429073 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.429018    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fdlm\" (UniqueName: \"kubernetes.io/projected/6d32df70-08f9-494a-81d5-868f1511dcb4-kube-api-access-5fdlm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.429073 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.429046    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d32df70-08f9-494a-81d5-868f1511dcb4-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.429166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.429077    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d32df70-08f9-494a-81d5-868f1511dcb4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.429166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.429113    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d32df70-08f9-494a-81d5-868f1511dcb4-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.475204 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.475177    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:35.529810 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.529783    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d32df70-08f9-494a-81d5-868f1511dcb4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.529922 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.529821    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fdlm\" (UniqueName: \"kubernetes.io/projected/6d32df70-08f9-494a-81d5-868f1511dcb4-kube-api-access-5fdlm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.529922 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.529842    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d32df70-08f9-494a-81d5-868f1511dcb4-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.529922 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.529878    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d32df70-08f9-494a-81d5-868f1511dcb4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.529922 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.529893    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d32df70-08f9-494a-81d5-868f1511dcb4-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.530139 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.529955    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d32df70-08f9-494a-81d5-868f1511dcb4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.530323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.530303    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d32df70-08f9-494a-81d5-868f1511dcb4-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.530398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.530339    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d32df70-08f9-494a-81d5-868f1511dcb4-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.530398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.530358    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d32df70-08f9-494a-81d5-868f1511dcb4-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.532212 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.532191    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d32df70-08f9-494a-81d5-868f1511dcb4-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.532605 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.532579    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d32df70-08f9-494a-81d5-868f1511dcb4-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.539850 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.539828    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fdlm\" (UniqueName: \"kubernetes.io/projected/6d32df70-08f9-494a-81d5-868f1511dcb4-kube-api-access-5fdlm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj\" (UID: \"6d32df70-08f9-494a-81d5-868f1511dcb4\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.657575 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.657499    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:35.959742 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:35.959711    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"]
Oct 30 18:46:35.963204 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:46:35.963172    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d32df70_08f9_494a_81d5_868f1511dcb4.slice/crio-e0aaf20e5a85225cdc3d4b329a81aa208d95004b361221dad20b7bf83b302085 WatchSource:0}: Error finding container e0aaf20e5a85225cdc3d4b329a81aa208d95004b361221dad20b7bf83b302085: Status 404 returned error can't find the container with id e0aaf20e5a85225cdc3d4b329a81aa208d95004b361221dad20b7bf83b302085
Oct 30 18:46:36.044612 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:36.044576    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj" event={"ID":"6d32df70-08f9-494a-81d5-868f1511dcb4","Type":"ContainerStarted","Data":"e0aaf20e5a85225cdc3d4b329a81aa208d95004b361221dad20b7bf83b302085"}
Oct 30 18:46:38.057832 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:38.057751    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj" event={"ID":"6d32df70-08f9-494a-81d5-868f1511dcb4","Type":"ContainerStarted","Data":"641831791ce2c06e10e2df772b65d7e613ec2d2be0443c04b0f1f1c35e5d6970"}
Oct 30 18:46:38.078350 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:38.078295    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj" podStartSLOduration=1.276939233 podStartE2EDuration="3.078281234s" totalImagesPullingTime="1.801342001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:38.076792027 +0000 UTC m=+619.793137102" watchObservedRunningTime="2026-10-30 18:46:38.078281234 +0000 UTC m=+619.794626309"
Oct 30 18:46:38.334012 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:38.333925    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:38.658510 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:38.658482    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:41.291936 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.291906    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"]
Oct 30 18:46:41.298746 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.298717    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.305331 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.305303    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 30 18:46:41.306650 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.306622    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"]
Oct 30 18:46:41.378582 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.378549    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7545f38-34b7-4c0d-9887-1a065c2231b0-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.378726 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.378608    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7545f38-34b7-4c0d-9887-1a065c2231b0-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.378726 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.378671    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7545f38-34b7-4c0d-9887-1a065c2231b0-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.378726 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.378706    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7545f38-34b7-4c0d-9887-1a065c2231b0-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.378726 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.378724    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bd87\" (UniqueName: \"kubernetes.io/projected/b7545f38-34b7-4c0d-9887-1a065c2231b0-kube-api-access-5bd87\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.378860 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.378742    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7545f38-34b7-4c0d-9887-1a065c2231b0-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.428449 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.428417    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:41.479597 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.479568    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7545f38-34b7-4c0d-9887-1a065c2231b0-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.479731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.479633    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7545f38-34b7-4c0d-9887-1a065c2231b0-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.479731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.479649    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7545f38-34b7-4c0d-9887-1a065c2231b0-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.479731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.479669    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7545f38-34b7-4c0d-9887-1a065c2231b0-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.479731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.479684    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bd87\" (UniqueName: \"kubernetes.io/projected/b7545f38-34b7-4c0d-9887-1a065c2231b0-kube-api-access-5bd87\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.479731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.479710    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7545f38-34b7-4c0d-9887-1a065c2231b0-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.480080 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.480059    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7545f38-34b7-4c0d-9887-1a065c2231b0-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.480196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.480158    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7545f38-34b7-4c0d-9887-1a065c2231b0-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.480196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.480181    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7545f38-34b7-4c0d-9887-1a065c2231b0-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.482321 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.482294    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7545f38-34b7-4c0d-9887-1a065c2231b0-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.482503 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.482485    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7545f38-34b7-4c0d-9887-1a065c2231b0-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.490345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.490320    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bd87\" (UniqueName: \"kubernetes.io/projected/b7545f38-34b7-4c0d-9887-1a065c2231b0-kube-api-access-5bd87\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-68f2m\" (UID: \"b7545f38-34b7-4c0d-9887-1a065c2231b0\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.615601 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.615541    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:41.825423 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.825388    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"]
Oct 30 18:46:41.830129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.830085    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.837196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.837160    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 18:46:41.843874 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.842067    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"]
Oct 30 18:46:41.883988 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.883924    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvh9r\" (UniqueName: \"kubernetes.io/projected/af327665-1858-4fc9-a89a-772751ad7921-kube-api-access-zvh9r\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.883988 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.883970    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af327665-1858-4fc9-a89a-772751ad7921-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.884208 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.884044    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af327665-1858-4fc9-a89a-772751ad7921-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.884208 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.884127    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af327665-1858-4fc9-a89a-772751ad7921-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.884208 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.884195    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af327665-1858-4fc9-a89a-772751ad7921-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.884331 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.884214    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af327665-1858-4fc9-a89a-772751ad7921-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.900512 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.900489    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"]
Oct 30 18:46:41.903927 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:46:41.903896    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7545f38_34b7_4c0d_9887_1a065c2231b0.slice/crio-599b3cd0679fc99a98f08f0f17e3045fbba1cdb2245bcc68d356ac22d549fb83 WatchSource:0}: Error finding container 599b3cd0679fc99a98f08f0f17e3045fbba1cdb2245bcc68d356ac22d549fb83: Status 404 returned error can't find the container with id 599b3cd0679fc99a98f08f0f17e3045fbba1cdb2245bcc68d356ac22d549fb83
Oct 30 18:46:41.985245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.985219    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af327665-1858-4fc9-a89a-772751ad7921-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.985369 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.985261    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af327665-1858-4fc9-a89a-772751ad7921-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.985369 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.985282    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af327665-1858-4fc9-a89a-772751ad7921-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.985369 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.985319    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af327665-1858-4fc9-a89a-772751ad7921-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.985369 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.985345    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af327665-1858-4fc9-a89a-772751ad7921-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.985545 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.985414    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvh9r\" (UniqueName: \"kubernetes.io/projected/af327665-1858-4fc9-a89a-772751ad7921-kube-api-access-zvh9r\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.985703 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.985655    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af327665-1858-4fc9-a89a-772751ad7921-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.985821 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.985726    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af327665-1858-4fc9-a89a-772751ad7921-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.985871 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.985822    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af327665-1858-4fc9-a89a-772751ad7921-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.987691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.987674    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af327665-1858-4fc9-a89a-772751ad7921-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:41.987880 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:41.987864    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af327665-1858-4fc9-a89a-772751ad7921-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:42.004475 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:42.004453    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvh9r\" (UniqueName: \"kubernetes.io/projected/af327665-1858-4fc9-a89a-772751ad7921-kube-api-access-zvh9r\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-npxnt\" (UID: \"af327665-1858-4fc9-a89a-772751ad7921\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:42.018458 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:42.018437    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:42.077520 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:42.077498    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m" event={"ID":"b7545f38-34b7-4c0d-9887-1a065c2231b0","Type":"ContainerStarted","Data":"599b3cd0679fc99a98f08f0f17e3045fbba1cdb2245bcc68d356ac22d549fb83"}
Oct 30 18:46:42.150864 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:42.150809    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:42.441478 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:42.441424    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"]
Oct 30 18:46:42.443731 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:46:42.443691    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf327665_1858_4fc9_a89a_772751ad7921.slice/crio-8d899dfefb4e3c4ab7123b253416cd6dd31f7eca4bcf360770a134ed3bbd241d WatchSource:0}: Error finding container 8d899dfefb4e3c4ab7123b253416cd6dd31f7eca4bcf360770a134ed3bbd241d: Status 404 returned error can't find the container with id 8d899dfefb4e3c4ab7123b253416cd6dd31f7eca4bcf360770a134ed3bbd241d
Oct 30 18:46:43.086831 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:43.086751    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt" event={"ID":"af327665-1858-4fc9-a89a-772751ad7921","Type":"ContainerStarted","Data":"201145abb813eab71a4c4cd9d8808586ae08bdc0945bc9a3bd5cd0e93eed1990"}
Oct 30 18:46:43.086831 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:43.086809    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt" event={"ID":"af327665-1858-4fc9-a89a-772751ad7921","Type":"ContainerStarted","Data":"8d899dfefb4e3c4ab7123b253416cd6dd31f7eca4bcf360770a134ed3bbd241d"}
Oct 30 18:46:43.088619 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:43.088593    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m" event={"ID":"b7545f38-34b7-4c0d-9887-1a065c2231b0","Type":"ContainerStarted","Data":"5c52672f5c4ab108ed2f69e253ae181c8ad1997af8c656076e1c398eee4e1a50"}
Oct 30 18:46:43.092918 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:43.092892    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:43.106829 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:43.106791    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt" podStartSLOduration=1.8301958310000002 podStartE2EDuration="2.106780918s" totalImagesPullingTime="276.585087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:43.104898452 +0000 UTC m=+624.821243540" watchObservedRunningTime="2026-10-30 18:46:43.106780918 +0000 UTC m=+624.823125994"
Oct 30 18:46:43.127012 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:43.126974    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m" podStartSLOduration=1.8403526559999999 podStartE2EDuration="2.126962133s" totalImagesPullingTime="286.609477ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:43.125653122 +0000 UTC m=+624.841998198" watchObservedRunningTime="2026-10-30 18:46:43.126962133 +0000 UTC m=+624.843307209"
Oct 30 18:46:43.151887 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:43.151865    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:43.303484 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:46:43.303450    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:46:43.615982 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:43.615948    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:44.453722 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.453693    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-qht75"]
Oct 30 18:46:44.456960 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.456939    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.464380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.464143    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Oct 30 18:46:44.471047 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.471019    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-qht75"]
Oct 30 18:46:44.506255 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.506224    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd42a605-1c88-4212-9199-1f3572a08984-home\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.506255 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.506253    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd42a605-1c88-4212-9199-1f3572a08984-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.506548 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.506531    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd42a605-1c88-4212-9199-1f3572a08984-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.506623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.506579    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgxk8\" (UniqueName: \"kubernetes.io/projected/bd42a605-1c88-4212-9199-1f3572a08984-kube-api-access-kgxk8\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.506677 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.506648    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd42a605-1c88-4212-9199-1f3572a08984-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.506790 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.506773    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd42a605-1c88-4212-9199-1f3572a08984-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.603462 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.603404    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:44.607323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.607296    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd42a605-1c88-4212-9199-1f3572a08984-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.607476 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.607345    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd42a605-1c88-4212-9199-1f3572a08984-home\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.607476 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.607365    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd42a605-1c88-4212-9199-1f3572a08984-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.607476 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.607411    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd42a605-1c88-4212-9199-1f3572a08984-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.607476 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.607437    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgxk8\" (UniqueName: \"kubernetes.io/projected/bd42a605-1c88-4212-9199-1f3572a08984-kube-api-access-kgxk8\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.607476 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.607454    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd42a605-1c88-4212-9199-1f3572a08984-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.607798 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.607769    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd42a605-1c88-4212-9199-1f3572a08984-home\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.607862 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.607842    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd42a605-1c88-4212-9199-1f3572a08984-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.607923 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.607845    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd42a605-1c88-4212-9199-1f3572a08984-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.610028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.610002    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd42a605-1c88-4212-9199-1f3572a08984-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.610392 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.610356    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd42a605-1c88-4212-9199-1f3572a08984-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.617878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.617858    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgxk8\" (UniqueName: \"kubernetes.io/projected/bd42a605-1c88-4212-9199-1f3572a08984-kube-api-access-kgxk8\") pod \"echo-simulator-kserve-64b6f8f96c-qht75\" (UID: \"bd42a605-1c88-4212-9199-1f3572a08984\") " pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:44.773446 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:44.773380    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:45.054877 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:45.054854    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-qht75"]
Oct 30 18:46:45.056646 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:46:45.056614    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd42a605_1c88_4212_9199_1f3572a08984.slice/crio-aff8c818ce11ca3b06aa1abaeb9fa2492d7b0a92b492b37e8a4f8ba2c1946a2f WatchSource:0}: Error finding container aff8c818ce11ca3b06aa1abaeb9fa2492d7b0a92b492b37e8a4f8ba2c1946a2f: Status 404 returned error can't find the container with id aff8c818ce11ca3b06aa1abaeb9fa2492d7b0a92b492b37e8a4f8ba2c1946a2f
Oct 30 18:46:45.098448 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:45.098383    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-qht75" event={"ID":"bd42a605-1c88-4212-9199-1f3572a08984","Type":"ContainerStarted","Data":"aff8c818ce11ca3b06aa1abaeb9fa2492d7b0a92b492b37e8a4f8ba2c1946a2f"}
Oct 30 18:46:45.200243 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:45.200214    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:45.658126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:45.657779    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:45.672375 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:45.672351    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:45.755033 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:45.755003    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:46.119748 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:46.119674    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj"
Oct 30 18:46:47.112856 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:47.112821    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-qht75" event={"ID":"bd42a605-1c88-4212-9199-1f3572a08984","Type":"ContainerStarted","Data":"f2f24beed8a990bedc99f4d682868b331cf749742032f0f3b42ed88bce2452c8"}
Oct 30 18:46:47.161017 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:47.160957    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-qht75" podStartSLOduration=1.287917547 podStartE2EDuration="3.160937996s" totalImagesPullingTime="1.873020449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:47.15928174 +0000 UTC m=+628.875626844" watchObservedRunningTime="2026-10-30 18:46:47.160937996 +0000 UTC m=+628.877283084"
Oct 30 18:46:47.774406 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:47.774373    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:51.616427 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:51.616390    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:51.634995 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:51.634969    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:52.148722 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:52.148691    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-68f2m"
Oct 30 18:46:52.151539 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:52.151514    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:52.164035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:52.164012    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:53.154137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:53.154109    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-npxnt"
Oct 30 18:46:54.773782 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:54.773752    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:54.786141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:54.786117    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:55.163568 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:55.163542    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-qht75"
Oct 30 18:46:56.791130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.791074    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"]
Oct 30 18:46:56.795222 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.795199    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:56.803705 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.803685    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-sj2rm\""
Oct 30 18:46:56.810699 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.810675    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"]
Oct 30 18:46:56.916349 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.916317    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ba4746-17cf-46a6-8517-720d4fbdb75a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:56.916349 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.916353    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-istio-envoy\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:56.916519 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.916378    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-workload-certs\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:56.916519 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.916425    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-istio-data\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:56.916581 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.916517    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-workload-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:56.916581 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.916543    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-credential-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:56.916581 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.916560    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ba4746-17cf-46a6-8517-720d4fbdb75a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:56.916680 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.916611    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ba4746-17cf-46a6-8517-720d4fbdb75a-istio-token\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:56.916680 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:56.916631    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqvj4\" (UniqueName: \"kubernetes.io/projected/63ba4746-17cf-46a6-8517-720d4fbdb75a-kube-api-access-lqvj4\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.017630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.017592    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ba4746-17cf-46a6-8517-720d4fbdb75a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.017630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.017639    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-istio-envoy\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.017801 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.017706    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-workload-certs\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.017801 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.017746    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-istio-data\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.017801 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.017770    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-workload-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.017801 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.017797    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-credential-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.017975 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.017822    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ba4746-17cf-46a6-8517-720d4fbdb75a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.017975 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.017853    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ba4746-17cf-46a6-8517-720d4fbdb75a-istio-token\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.017975 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.017876    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqvj4\" (UniqueName: \"kubernetes.io/projected/63ba4746-17cf-46a6-8517-720d4fbdb75a-kube-api-access-lqvj4\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.018222 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.018186    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-istio-data\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.018296 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.018206    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-workload-certs\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.018296 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.018282    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-workload-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.018380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.018303    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-credential-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.018698 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.018681    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ba4746-17cf-46a6-8517-720d4fbdb75a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.019986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.019964    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ba4746-17cf-46a6-8517-720d4fbdb75a-istio-envoy\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.020314 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.020295    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ba4746-17cf-46a6-8517-720d4fbdb75a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.028975 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.028952    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ba4746-17cf-46a6-8517-720d4fbdb75a-istio-token\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.029206 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.029189    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqvj4\" (UniqueName: \"kubernetes.io/projected/63ba4746-17cf-46a6-8517-720d4fbdb75a-kube-api-access-lqvj4\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-78fkg\" (UID: \"63ba4746-17cf-46a6-8517-720d4fbdb75a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.108145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.108067    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:46:57.402411 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:57.402302    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"]
Oct 30 18:46:57.404690 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:46:57.404653    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ba4746_17cf_46a6_8517_720d4fbdb75a.slice/crio-50d078a7c47a9321faa362154bd2ef5bdd7305c70bd33e0024e17740aa4f8cfe WatchSource:0}: Error finding container 50d078a7c47a9321faa362154bd2ef5bdd7305c70bd33e0024e17740aa4f8cfe: Status 404 returned error can't find the container with id 50d078a7c47a9321faa362154bd2ef5bdd7305c70bd33e0024e17740aa4f8cfe
Oct 30 18:46:58.168019 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:46:58.167984    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg" event={"ID":"63ba4746-17cf-46a6-8517-720d4fbdb75a","Type":"ContainerStarted","Data":"50d078a7c47a9321faa362154bd2ef5bdd7305c70bd33e0024e17740aa4f8cfe"}
Oct 30 18:47:00.699035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:00.699006    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:00.699325 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:00.699083    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:00.699325 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:00.699138    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:01.192908 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:01.192869    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg" event={"ID":"63ba4746-17cf-46a6-8517-720d4fbdb75a","Type":"ContainerStarted","Data":"8083c4277c6c1fc65fad0f076945dd97994c53571e57c3e9ba9f51bffb0fde80"}
Oct 30 18:47:01.215635 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:01.215579    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg" podStartSLOduration=1.924075017 podStartE2EDuration="5.215566866s" totalImagesPullingTime="3.291491849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:01.214543262 +0000 UTC m=+642.930888357" watchObservedRunningTime="2026-10-30 18:47:01.215566866 +0000 UTC m=+642.931911941"
Oct 30 18:47:01.883772 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:01.883734    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:02.087072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:02.087035    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:02.109210 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:02.109175    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:47:02.110849 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:02.110821    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-78fkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 30 18:47:02.110989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:02.110875    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg" podUID="63ba4746-17cf-46a6-8517-720d4fbdb75a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 30 18:47:03.108995 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:03.108958    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-78fkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 30 18:47:03.109511 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:03.109017    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg" podUID="63ba4746-17cf-46a6-8517-720d4fbdb75a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 30 18:47:03.464122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:03.462354    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:03.674046 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:03.674005    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:04.108428 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:04.108391    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-78fkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 30 18:47:04.108575 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:04.108454    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg" podUID="63ba4746-17cf-46a6-8517-720d4fbdb75a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 30 18:47:05.109001 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:05.108960    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-78fkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 30 18:47:05.109398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:05.109027    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg" podUID="63ba4746-17cf-46a6-8517-720d4fbdb75a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 30 18:47:06.109280 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:06.109227    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-78fkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 30 18:47:06.109777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:06.109295    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg" podUID="63ba4746-17cf-46a6-8517-720d4fbdb75a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 30 18:47:07.108688 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:07.108641    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-78fkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 30 18:47:07.108876 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:07.108718    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg" podUID="63ba4746-17cf-46a6-8517-720d4fbdb75a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 30 18:47:08.316644 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:08.316595    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:08.898346 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:08.898317    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:47:08.898779 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:08.898721    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:47:08.899865 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:08.899845    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-78fkg"
Oct 30 18:47:13.301719 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:47:13.301681    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:47:32.858493 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:32.858409    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:33.076021 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.075979    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:33.266004 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.265972    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:33.653850 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.653749    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:33.707252 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.704503    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"]
Oct 30 18:47:33.717642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.717615    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.724623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.724595    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"]
Oct 30 18:47:33.728368 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.728346    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e4242163-openshift-default-dockercfg-8xvxw\""
Oct 30 18:47:33.800180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.800130    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"]
Oct 30 18:47:33.804598 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.804575    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:33.828222 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.828199    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"]
Oct 30 18:47:33.829178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.829155    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-226ecbcb-openshift-default-dockercfg-z2wv2\""
Oct 30 18:47:33.862541 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.862516    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-credential-socket\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.862851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.862554    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp4xg\" (UniqueName: \"kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-kube-api-access-wp4xg\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.862851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.862574    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-socket\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.862851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.862589    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-envoy\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.862851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.862672    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-istio-token\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.862851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.862754    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b48776-3a2c-423c-b695-89de08958a13-istiod-ca-cert\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.862851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.862782    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-data\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.862851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.862836    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-certs\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.863077 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.862872    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b48776-3a2c-423c-b695-89de08958a13-istio-podinfo\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.888633 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.888607    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"]
Oct 30 18:47:33.892574 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.892551    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:33.905285 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.905220    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:33.908156 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.908130    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"]
Oct 30 18:47:33.927901 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.927878    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-6156d5d6-gw-openshift-default-dockercfg-s6p4n\""
Oct 30 18:47:33.963824 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.963790    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-certs\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:33.963824 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.963823    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-istio-token\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:33.964010 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.963846    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b48776-3a2c-423c-b695-89de08958a13-istio-podinfo\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.964010 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.963868    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-data\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:33.964010 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.963942    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-socket\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.964010 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.963958    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-certs\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:33.964524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.964048    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-socket\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:33.964524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.964126    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-token\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:33.964524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.964160    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-envoy\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:33.964524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.964188    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-certs\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.964524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.964226    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/703c1835-5274-40b5-a967-096296705e15-istio-podinfo\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:33.964524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.964263    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwx59\" (UniqueName: \"kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-kube-api-access-fwx59\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:33.964524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.964290    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/703c1835-5274-40b5-a967-096296705e15-istiod-ca-cert\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:33.964524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.964369    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-socket\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.964524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.964487    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-envoy\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.965440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.964552    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b48776-3a2c-423c-b695-89de08958a13-istiod-ca-cert\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.965440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.964634    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-certs\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.965440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.964973    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-credential-socket\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.965440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.965039    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r5lz\" (UniqueName: \"kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-kube-api-access-9r5lz\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:33.965659 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.965068    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-podinfo\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:33.965659 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.965504    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp4xg\" (UniqueName: \"kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-kube-api-access-wp4xg\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.966068 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.966046    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-istio-token\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.966339 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.966116    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-envoy\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:33.966339 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.966146    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-data\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.966339 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.966172    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-socket\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:33.966600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.966413    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istiod-ca-cert\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:33.966600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.966442    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-credential-socket\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:33.966600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.966466    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-credential-socket\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:33.966600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.966357    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-credential-socket\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.966600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.965887    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b48776-3a2c-423c-b695-89de08958a13-istiod-ca-cert\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.966600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.966556    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-data\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:33.966922 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.966701    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-data\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.967517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.967490    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b48776-3a2c-423c-b695-89de08958a13-istio-podinfo\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.967603 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.967491    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-envoy\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.977359 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.977335    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-istio-token\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:33.977444 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:33.977402    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp4xg\" (UniqueName: \"kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-kube-api-access-wp4xg\") pod \"e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") " pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:34.035158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.035135    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:34.069754 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.069705    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-envoy\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.070007 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.069984    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-socket\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.070168 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.070152    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istiod-ca-cert\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.070464 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.070432    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-credential-socket\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.070552 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.070505    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-credential-socket\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.070691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.070542    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-data\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.070843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.070714    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-certs\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.070921 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.070866    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-istio-token\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.071031 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.071009    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-data\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.071226 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.071202    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-socket\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.071473 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.071451    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-data\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.071690 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.071670    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istiod-ca-cert\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.071798 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.071683    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-credential-socket\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.071909 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.071889    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-credential-socket\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.072245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.072002    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-certs\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.072245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.072069    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-socket\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.072245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.072161    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-token\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.072245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.072210    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-envoy\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.072245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.072248    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/703c1835-5274-40b5-a967-096296705e15-istio-podinfo\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.072517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.072300    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwx59\" (UniqueName: \"kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-kube-api-access-fwx59\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.072517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.072321    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/703c1835-5274-40b5-a967-096296705e15-istiod-ca-cert\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.072517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.072401    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r5lz\" (UniqueName: \"kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-kube-api-access-9r5lz\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.072517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.072426    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-podinfo\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.073144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.073077    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-certs\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.073378 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.073357    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-certs\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.073529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.073429    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-data\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.073659 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.073613    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-socket\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.073759 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.073672    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/703c1835-5274-40b5-a967-096296705e15-istiod-ca-cert\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.075625 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.075580    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-envoy\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.076994 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.076482    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/703c1835-5274-40b5-a967-096296705e15-istio-podinfo\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.076994 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.076936    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-podinfo\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.077174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.077148    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-envoy\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.087113 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.087074    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-istio-token\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.087861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.087836    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r5lz\" (UniqueName: \"kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-kube-api-access-9r5lz\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.088638 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.088605    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwx59\" (UniqueName: \"kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-kube-api-access-fwx59\") pod \"e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") " pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.089124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.089079    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-token\") pod \"e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") " pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.112488 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.112262    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:34.120872 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.120461    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:34.208692 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.208660    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:34.297772 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.297741    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:34.331070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.331015    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"]
Oct 30 18:47:34.333486 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:47:34.333463    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b48776_3a2c_423c_b695_89de08958a13.slice/crio-58baf7898eea65ac8ee984eaa6fcf708157a4aea821aa1854be25c80c359ae30 WatchSource:0}: Error finding container 58baf7898eea65ac8ee984eaa6fcf708157a4aea821aa1854be25c80c359ae30: Status 404 returned error can't find the container with id 58baf7898eea65ac8ee984eaa6fcf708157a4aea821aa1854be25c80c359ae30
Oct 30 18:47:34.335482 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.335451    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:34.335591 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.335530    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:34.335591 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.335578    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:34.345259 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.345230    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" event={"ID":"c4b48776-3a2c-423c-b695-89de08958a13","Type":"ContainerStarted","Data":"58baf7898eea65ac8ee984eaa6fcf708157a4aea821aa1854be25c80c359ae30"}
Oct 30 18:47:34.412252 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.412222    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"]
Oct 30 18:47:34.418267 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:47:34.418218    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56fbb51c_5be2_4b4e_90eb_fe3e81060f80.slice/crio-4fc26ef3c5f40999edab5ff962fba1941f54c9419ae543e73a233cf32f48b075 WatchSource:0}: Error finding container 4fc26ef3c5f40999edab5ff962fba1941f54c9419ae543e73a233cf32f48b075: Status 404 returned error can't find the container with id 4fc26ef3c5f40999edab5ff962fba1941f54c9419ae543e73a233cf32f48b075
Oct 30 18:47:34.420668 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.420634    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:34.420786 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.420718    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:34.420786 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.420765    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:34.495739 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.494216    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"]
Oct 30 18:47:34.497310 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.496854    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:34.497310 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.496936    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:34.497310 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.496981    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:34.836249 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:34.836159    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:35.251071 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:35.251032    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:35.352564 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:35.352531    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" event={"ID":"c4b48776-3a2c-423c-b695-89de08958a13","Type":"ContainerStarted","Data":"16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228"}
Oct 30 18:47:35.354203 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:35.354178    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" event={"ID":"703c1835-5274-40b5-a967-096296705e15","Type":"ContainerStarted","Data":"baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c"}
Oct 30 18:47:35.354336 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:35.354223    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" event={"ID":"703c1835-5274-40b5-a967-096296705e15","Type":"ContainerStarted","Data":"4c89d3949423ad5266c0c46c462b22ee3af8447aec8c1004bda0866a8780d8e9"}
Oct 30 18:47:35.355875 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:35.355855    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" event={"ID":"56fbb51c-5be2-4b4e-90eb-fe3e81060f80","Type":"ContainerStarted","Data":"e6545be4eed74c3772676c28b35a21db0be7a39eaecbb8d395273fd37a1a779c"}
Oct 30 18:47:35.355973 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:35.355890    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" event={"ID":"56fbb51c-5be2-4b4e-90eb-fe3e81060f80","Type":"ContainerStarted","Data":"4fc26ef3c5f40999edab5ff962fba1941f54c9419ae543e73a233cf32f48b075"}
Oct 30 18:47:35.374758 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:35.374720    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" podStartSLOduration=2.374708235 podStartE2EDuration="2.374708235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:35.373229801 +0000 UTC m=+677.089574877" watchObservedRunningTime="2026-10-30 18:47:35.374708235 +0000 UTC m=+677.091053311"
Oct 30 18:47:35.392752 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:35.392696    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" podStartSLOduration=2.392658168 podStartE2EDuration="2.392658168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:35.39200637 +0000 UTC m=+677.108351436" watchObservedRunningTime="2026-10-30 18:47:35.392658168 +0000 UTC m=+677.109003246"
Oct 30 18:47:35.411793 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:35.411755    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" podStartSLOduration=2.411743126 podStartE2EDuration="2.411743126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:35.410689781 +0000 UTC m=+677.127034854" watchObservedRunningTime="2026-10-30 18:47:35.411743126 +0000 UTC m=+677.128088202"
Oct 30 18:47:35.431914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:35.431889    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:36.036021 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:36.035986    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:36.041735 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:36.041711    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:36.121157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:36.121129    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:36.126051 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:36.126026    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:36.209137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:36.209079    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:36.214143 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:36.214115    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:36.361441 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:36.361360    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:36.361441 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:36.361421    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:36.361936 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:36.361822    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:36.362445 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:36.362421    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:36.362591 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:36.362576    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:36.362630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:36.362623    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:37.044887 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.044851    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:37.279021 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.278986    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:37.559921 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.559884    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:37.583305 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.583273    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"]
Oct 30 18:47:37.587252 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.587233    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.592563 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.592544    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-6156d5d6-serving-cert\""
Oct 30 18:47:37.607156 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.607135    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"]
Oct 30 18:47:37.710861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.710825    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqhcl\" (UniqueName: \"kubernetes.io/projected/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-kube-api-access-tqhcl\") pod \"maas-api-e2e-migrate-6156d5d6-55454bd86-262hk\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.711036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.710878    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-maas-api-tls\") pod \"maas-api-e2e-migrate-6156d5d6-55454bd86-262hk\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.711036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.710945    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-metrics-certs\") pod \"maas-api-e2e-migrate-6156d5d6-55454bd86-262hk\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.711036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.711017    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-openshift-service-ca\") pod \"maas-api-e2e-migrate-6156d5d6-55454bd86-262hk\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.809421 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.809365    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:37.812521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.812468    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-metrics-certs\") pod \"maas-api-e2e-migrate-6156d5d6-55454bd86-262hk\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.812605 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.812540    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-openshift-service-ca\") pod \"maas-api-e2e-migrate-6156d5d6-55454bd86-262hk\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.812605 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.812573    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqhcl\" (UniqueName: \"kubernetes.io/projected/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-kube-api-access-tqhcl\") pod \"maas-api-e2e-migrate-6156d5d6-55454bd86-262hk\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.812605 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.812594    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-maas-api-tls\") pod \"maas-api-e2e-migrate-6156d5d6-55454bd86-262hk\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.813313 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.813289    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-openshift-service-ca\") pod \"maas-api-e2e-migrate-6156d5d6-55454bd86-262hk\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.815141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.815118    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-metrics-certs\") pod \"maas-api-e2e-migrate-6156d5d6-55454bd86-262hk\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.815250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.815202    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-maas-api-tls\") pod \"maas-api-e2e-migrate-6156d5d6-55454bd86-262hk\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.833440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.833397    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqhcl\" (UniqueName: \"kubernetes.io/projected/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-kube-api-access-tqhcl\") pod \"maas-api-e2e-migrate-6156d5d6-55454bd86-262hk\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:37.901843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:37.901808    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:38.096089 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:38.096007    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:38.205534 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:38.205500    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"]
Oct 30 18:47:38.209446 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:47:38.209413    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd445b11_79cb_4d2a_a8b5_3b386ea146e8.slice/crio-e260e56b7a43ac77ff7be66e8f0762ee709a19ba0ab087ebaaadea860716ae0e WatchSource:0}: Error finding container e260e56b7a43ac77ff7be66e8f0762ee709a19ba0ab087ebaaadea860716ae0e: Status 404 returned error can't find the container with id e260e56b7a43ac77ff7be66e8f0762ee709a19ba0ab087ebaaadea860716ae0e
Oct 30 18:47:38.211278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:38.211258    4925 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:47:38.338841 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:38.338803    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:38.373631 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:38.373558    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk" event={"ID":"dd445b11-79cb-4d2a-a8b5-3b386ea146e8","Type":"ContainerStarted","Data":"e260e56b7a43ac77ff7be66e8f0762ee709a19ba0ab087ebaaadea860716ae0e"}
Oct 30 18:47:38.809475 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:38.809293    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:39.048038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:39.047936    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:39.306809 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:39.306708    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:39.379814 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:39.379780    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk" event={"ID":"dd445b11-79cb-4d2a-a8b5-3b386ea146e8","Type":"ContainerStarted","Data":"4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd"}
Oct 30 18:47:39.379978 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:39.379955    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:39.402609 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:39.402558    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk" podStartSLOduration=1.907503353 podStartE2EDuration="2.402546784s" totalImagesPullingTime="495.043431ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:39.401381428 +0000 UTC m=+681.117726507" watchObservedRunningTime="2026-10-30 18:47:39.402546784 +0000 UTC m=+681.118891861"
Oct 30 18:47:39.646400 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:39.646319    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:39.909172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:39.909139    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:40.186205 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:40.184477    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:40.435600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:40.435568    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:40.950762 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:40.950726    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"]
Oct 30 18:47:40.954948 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:40.954924    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:40.963622 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:40.963578    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-226ecbcb-serving-cert\""
Oct 30 18:47:40.966976 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:40.966944    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"]
Oct 30 18:47:41.052495 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.052467    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfsqn\" (UniqueName: \"kubernetes.io/projected/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-kube-api-access-hfsqn\") pod \"maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.052653 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.052507    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-openshift-service-ca\") pod \"maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.052653 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.052544    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-metrics-certs\") pod \"maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.052751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.052693    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-maas-api-tls\") pod \"maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.096074 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.096039    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:41.153355 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.153319    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfsqn\" (UniqueName: \"kubernetes.io/projected/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-kube-api-access-hfsqn\") pod \"maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.153530 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.153375    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-openshift-service-ca\") pod \"maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.153530 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.153414    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-metrics-certs\") pod \"maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.153530 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.153503    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-maas-api-tls\") pod \"maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.154126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.154066    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-openshift-service-ca\") pod \"maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.156358 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.156335    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-maas-api-tls\") pod \"maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.156525 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.156511    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-metrics-certs\") pod \"maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.164553 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.164526    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfsqn\" (UniqueName: \"kubernetes.io/projected/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-kube-api-access-hfsqn\") pod \"maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.274887 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.274818    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:41.291274 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.291225    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:41.498974 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.498924    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:41.567938 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:41.567863    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"]
Oct 30 18:47:41.571191 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:47:41.571162    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda3d3e1_272d_4b12_9ef3_4a9092809f9e.slice/crio-ae8092fb254953c68cda5ca1a7edbbf5c6ad3bd56709fd7c071e90d50ca0d611 WatchSource:0}: Error finding container ae8092fb254953c68cda5ca1a7edbbf5c6ad3bd56709fd7c071e90d50ca0d611: Status 404 returned error can't find the container with id ae8092fb254953c68cda5ca1a7edbbf5c6ad3bd56709fd7c071e90d50ca0d611
Oct 30 18:47:42.152152 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:42.152120    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:42.402028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:42.401937    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg" event={"ID":"cda3d3e1-272d-4b12-9ef3-4a9092809f9e","Type":"ContainerStarted","Data":"5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7"}
Oct 30 18:47:42.402028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:42.402005    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg" event={"ID":"cda3d3e1-272d-4b12-9ef3-4a9092809f9e","Type":"ContainerStarted","Data":"ae8092fb254953c68cda5ca1a7edbbf5c6ad3bd56709fd7c071e90d50ca0d611"}
Oct 30 18:47:42.402298 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:42.402058    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:42.429239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:42.429193    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg" podStartSLOduration=1.990406994 podStartE2EDuration="2.429179979s" totalImagesPullingTime="438.772985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:42.427969263 +0000 UTC m=+684.144314350" watchObservedRunningTime="2026-10-30 18:47:42.429179979 +0000 UTC m=+684.145525074"
Oct 30 18:47:42.481120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:42.480643    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:42.746787 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:42.746737    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:43.125711 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.125636    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:43.322277 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:47:43.322214    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:47:43.377168 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.362634    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:43.491492 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.491157    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"]
Oct 30 18:47:43.496442 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.496317    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"]
Oct 30 18:47:43.496654 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.496412    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.501291 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.501262    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ee87bf29-serving-cert\""
Oct 30 18:47:43.608387 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.600699    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nhfn\" (UniqueName: \"kubernetes.io/projected/34f503b7-7b08-4d20-9f68-35a328fb8693-kube-api-access-6nhfn\") pod \"maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.608387 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.600765    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34f503b7-7b08-4d20-9f68-35a328fb8693-openshift-service-ca\") pod \"maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.608387 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.600823    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-maas-api-tls\") pod \"maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.608387 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.600882    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-metrics-certs\") pod \"maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.633549 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.633448    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:43.702355 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.702324    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-maas-api-tls\") pod \"maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.702559 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.702368    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-metrics-certs\") pod \"maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.702559 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.702445    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nhfn\" (UniqueName: \"kubernetes.io/projected/34f503b7-7b08-4d20-9f68-35a328fb8693-kube-api-access-6nhfn\") pod \"maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.702559 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.702477    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34f503b7-7b08-4d20-9f68-35a328fb8693-openshift-service-ca\") pod \"maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.703530 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.703511    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34f503b7-7b08-4d20-9f68-35a328fb8693-openshift-service-ca\") pod \"maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.705843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.705819    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-metrics-certs\") pod \"maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.706637 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.706621    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-maas-api-tls\") pod \"maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.711865 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.711845    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nhfn\" (UniqueName: \"kubernetes.io/projected/34f503b7-7b08-4d20-9f68-35a328fb8693-kube-api-access-6nhfn\") pod \"maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.812573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.812539    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:43.851027 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:43.850981    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:44.087777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:44.087741    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:44.129286 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:44.129256    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"]
Oct 30 18:47:44.134213 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:47:44.134146    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34f503b7_7b08_4d20_9f68_35a328fb8693.slice/crio-4620ac422b6bdcfecc1a1e6ff0269e4af0377c9bd9c148a61fd4917d8dfcbf3e WatchSource:0}: Error finding container 4620ac422b6bdcfecc1a1e6ff0269e4af0377c9bd9c148a61fd4917d8dfcbf3e: Status 404 returned error can't find the container with id 4620ac422b6bdcfecc1a1e6ff0269e4af0377c9bd9c148a61fd4917d8dfcbf3e
Oct 30 18:47:44.413026 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:44.412983    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk" event={"ID":"34f503b7-7b08-4d20-9f68-35a328fb8693","Type":"ContainerStarted","Data":"4620ac422b6bdcfecc1a1e6ff0269e4af0377c9bd9c148a61fd4917d8dfcbf3e"}
Oct 30 18:47:44.695176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:44.692615    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:45.390907 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:45.390880    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:47:45.420176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:45.420085    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk" event={"ID":"34f503b7-7b08-4d20-9f68-35a328fb8693","Type":"ContainerStarted","Data":"5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b"}
Oct 30 18:47:45.420790 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:45.420719    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:45.481121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:45.465325    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk" podStartSLOduration=2.037007403 podStartE2EDuration="2.465303647s" totalImagesPullingTime="428.296244ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:45.454315527 +0000 UTC m=+687.170660621" watchObservedRunningTime="2026-10-30 18:47:45.465303647 +0000 UTC m=+687.181648724"
Oct 30 18:47:45.881120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:45.880741    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:46.077121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:46.070591    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:46.262708 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:46.262673    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:47.345138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:47.345086    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:47.748503 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:47.747952    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"]
Oct 30 18:47:47.753563 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:47.752720    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"
Oct 30 18:47:47.818122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:47.757415    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"]
Oct 30 18:47:47.829297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:47.771554    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-f951dfa9-dockercfg-lgtfm\""
Oct 30 18:47:47.829297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:47.786305    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:47.829297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:47.827556    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-f951dfa9\""
Oct 30 18:47:47.936392 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:47.892142    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qztz5\" (UniqueName: \"kubernetes.io/projected/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-kube-api-access-qztz5\") pod \"payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2\" (UID: \"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"
Oct 30 18:47:47.936392 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:47.892224    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2\" (UID: \"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"
Oct 30 18:47:47.995124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:47.994792    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qztz5\" (UniqueName: \"kubernetes.io/projected/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-kube-api-access-qztz5\") pod \"payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2\" (UID: \"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"
Oct 30 18:47:47.995124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:47.994867    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2\" (UID: \"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"
Oct 30 18:47:47.996126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:47.995827    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2\" (UID: \"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"
Oct 30 18:47:48.019168 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:48.016136    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:48.025390 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:48.024788    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qztz5\" (UniqueName: \"kubernetes.io/projected/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-kube-api-access-qztz5\") pod \"payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2\" (UID: \"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"
Oct 30 18:47:48.142161 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:48.142010    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"
Oct 30 18:47:48.452850 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:48.452352    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:47:48.499127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:48.498532    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:48.574233 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:48.570580    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"]
Oct 30 18:47:48.878183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:48.878000    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:49.543877 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:49.543845    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2" event={"ID":"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e","Type":"ContainerStarted","Data":"5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45"}
Oct 30 18:47:49.544374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:49.543895    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2" event={"ID":"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e","Type":"ContainerStarted","Data":"a922d8d37caff377cbcfa46aa706252f4fbdf022e68a13d2166621e49fcfc901"}
Oct 30 18:47:49.544374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:49.543991    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"
Oct 30 18:47:49.564121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:49.562935    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2" podStartSLOduration=2.56291819 podStartE2EDuration="2.56291819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:49.562433006 +0000 UTC m=+691.278778083" watchObservedRunningTime="2026-10-30 18:47:49.56291819 +0000 UTC m=+691.279263267"
Oct 30 18:47:49.666119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:49.656323    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:50.578589 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:50.578535    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:51.478119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:51.466429    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:47:51.523949 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:51.523911    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:52.037633 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:52.037587    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:52.406293 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:52.406247    4925 patch_prober.go:29] interesting pod/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:47:52.406458 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:52.406323    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:47:52.538482 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:52.538392    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:53.669244 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:53.665808    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:54.569426 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:54.569388    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:54.584808 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:54.584772    4925 generic.go:356] "Generic (PLEG): container finished" podID="703c1835-5274-40b5-a967-096296705e15" containerID="baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c" exitCode=0
Oct 30 18:47:54.584955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:54.584846    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" event={"ID":"703c1835-5274-40b5-a967-096296705e15","Type":"ContainerDied","Data":"baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c"}
Oct 30 18:47:54.585364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:54.585344    4925 scope.go:122] "RemoveContainer" containerID="baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c"
Oct 30 18:47:54.587055 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:54.587025    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:54.587188 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:54.587120    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:54.587188 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:54.587167    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:55.598618 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:55.598576    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" event={"ID":"703c1835-5274-40b5-a967-096296705e15","Type":"ContainerStarted","Data":"fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6"}
Oct 30 18:47:55.877177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:55.873826    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:56.209546 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.209520    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:47:56.209850 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.209817    4925 patch_prober.go:29] interesting pod/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Oct 30 18:47:56.209958 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.209869    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Oct 30 18:47:56.609494 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.609315    4925 generic.go:356] "Generic (PLEG): container finished" podID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerID="e6545be4eed74c3772676c28b35a21db0be7a39eaecbb8d395273fd37a1a779c" exitCode=0
Oct 30 18:47:56.609494 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.609399    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" event={"ID":"56fbb51c-5be2-4b4e-90eb-fe3e81060f80","Type":"ContainerDied","Data":"e6545be4eed74c3772676c28b35a21db0be7a39eaecbb8d395273fd37a1a779c"}
Oct 30 18:47:56.610028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.609851    4925 scope.go:122] "RemoveContainer" containerID="e6545be4eed74c3772676c28b35a21db0be7a39eaecbb8d395273fd37a1a779c"
Oct 30 18:47:56.611455 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.611380    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:56.611576 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.611471    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:56.611576 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.611525    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:56.612691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.612663    4925 generic.go:356] "Generic (PLEG): container finished" podID="c4b48776-3a2c-423c-b695-89de08958a13" containerID="16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228" exitCode=0
Oct 30 18:47:56.612804 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.612740    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" event={"ID":"c4b48776-3a2c-423c-b695-89de08958a13","Type":"ContainerDied","Data":"16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228"}
Oct 30 18:47:56.613047 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.613029    4925 scope.go:122] "RemoveContainer" containerID="16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228"
Oct 30 18:47:56.614309 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.614274    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:56.614411 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.614390    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:56.614480 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.614445    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:56.994129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:56.993691    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:57.209344 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:57.209307    4925 patch_prober.go:29] interesting pod/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Oct 30 18:47:57.209540 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:57.209371    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Oct 30 18:47:57.635120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:57.634880    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" event={"ID":"c4b48776-3a2c-423c-b695-89de08958a13","Type":"ContainerStarted","Data":"d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329"}
Oct 30 18:47:57.644116 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:57.641199    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" event={"ID":"56fbb51c-5be2-4b4e-90eb-fe3e81060f80","Type":"ContainerStarted","Data":"bfe2ef3d78334eba77ede2c63957a11cd1d5b1174de97036364fd891711498fe"}
Oct 30 18:47:58.055136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:58.054631    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:47:58.055136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:58.054788    4925 patch_prober.go:29] interesting pod/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 30 18:47:58.055136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:58.054832    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 30 18:47:58.155124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:58.137841    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:47:58.155124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:58.137988    4925 patch_prober.go:29] interesting pod/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Oct 30 18:47:58.155124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:58.138033    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Oct 30 18:47:58.227319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:58.216288    4925 patch_prober.go:29] interesting pod/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Oct 30 18:47:58.227319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:58.216360    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Oct 30 18:47:58.389514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:58.389404    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:59.050156 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:59.049660    4925 patch_prober.go:29] interesting pod/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 30 18:47:59.050156 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:59.049729    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 30 18:47:59.132344 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:59.132308    4925 patch_prober.go:29] interesting pod/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Oct 30 18:47:59.133247 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:59.133213    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Oct 30 18:47:59.211498 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:59.211462    4925 patch_prober.go:29] interesting pod/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Oct 30 18:47:59.211796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:59.211766    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Oct 30 18:47:59.930965 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:47:59.930879    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:00.064138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:00.048513    4925 patch_prober.go:29] interesting pod/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 30 18:48:00.064138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:00.048573    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 30 18:48:00.132434 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:00.132399    4925 patch_prober.go:29] interesting pod/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Oct 30 18:48:00.158671 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:00.158628    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Oct 30 18:48:00.219284 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:00.219193    4925 patch_prober.go:29] interesting pod/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Oct 30 18:48:00.219529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:00.219499    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Oct 30 18:48:00.586388 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:00.586276    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"
Oct 30 18:48:01.050608 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:01.050559    4925 patch_prober.go:29] interesting pod/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 30 18:48:01.050817 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:01.050627    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 30 18:48:01.122906 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:01.122457    4925 patch_prober.go:29] interesting pod/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Oct 30 18:48:01.122906 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:01.122520    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Oct 30 18:48:01.219292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:01.219245    4925 patch_prober.go:29] interesting pod/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Oct 30 18:48:01.219608 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:01.219553    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Oct 30 18:48:01.681725 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:01.681686    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:02.036416 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:02.036329    4925 patch_prober.go:29] interesting pod/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 30 18:48:02.036594 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:02.036407    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 30 18:48:02.122239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:02.122042    4925 patch_prober.go:29] interesting pod/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Oct 30 18:48:02.122239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:02.122117    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Oct 30 18:48:02.214963 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:02.214924    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:48:02.216197 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:02.216175    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:48:02.217591 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:02.217573    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:48:02.712122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:02.710696    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:02.903299 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:02.903255    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:03.037449 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:03.037338    4925 patch_prober.go:29] interesting pod/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 30 18:48:03.037449 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:03.037426    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 30 18:48:03.119986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:03.119919    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:03.219443 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:03.219406    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:48:03.220792 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:03.220758    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:48:03.221731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:03.221709    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:48:03.945641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:03.945605    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:04.039710 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:04.039684    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:48:04.039969 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:04.039949    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:48:04.040593 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:04.040580    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:48:04.592137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:04.592108    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:05.273625 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.273589    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:05.674444 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.674401    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"]
Oct 30 18:48:05.674858 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.674729    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk" podUID="dd445b11-79cb-4d2a-a8b5-3b386ea146e8" containerName="maas-api" containerID="cri-o://4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd" gracePeriod=30
Oct 30 18:48:05.819353 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.819311    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:05.944884 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.944859    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:48:05.979467 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.979437    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-openshift-service-ca\") pod \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") "
Oct 30 18:48:05.979608 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.979473    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-maas-api-tls\") pod \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") "
Oct 30 18:48:05.979608 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.979527    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-metrics-certs\") pod \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") "
Oct 30 18:48:05.979608 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.979588    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-kube-api-access-tqhcl\" (UniqueName: \"kubernetes.io/projected/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-kube-api-access-tqhcl\") pod \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\" (UID: \"dd445b11-79cb-4d2a-a8b5-3b386ea146e8\") "
Oct 30 18:48:05.979880 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.979855    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-openshift-service-ca" pod "dd445b11-79cb-4d2a-a8b5-3b386ea146e8" (UID: "dd445b11-79cb-4d2a-a8b5-3b386ea146e8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:05.980005 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.979987    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:05.981968 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.981943    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-maas-api-tls" pod "dd445b11-79cb-4d2a-a8b5-3b386ea146e8" (UID: "dd445b11-79cb-4d2a-a8b5-3b386ea146e8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:05.982067 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.981999    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-metrics-certs" pod "dd445b11-79cb-4d2a-a8b5-3b386ea146e8" (UID: "dd445b11-79cb-4d2a-a8b5-3b386ea146e8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:05.982067 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:05.982030    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-kube-api-access-tqhcl" pod "dd445b11-79cb-4d2a-a8b5-3b386ea146e8" (UID: "dd445b11-79cb-4d2a-a8b5-3b386ea146e8"). InnerVolumeSpecName "kube-api-access-tqhcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:06.069371 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.069330    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:06.081580 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.081548    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:06.081706 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.081610    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:06.081706 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.081630    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqhcl\" (UniqueName: \"kubernetes.io/projected/dd445b11-79cb-4d2a-a8b5-3b386ea146e8-kube-api-access-tqhcl\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:06.167035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.167002    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"]
Oct 30 18:48:06.167327 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.167301    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy" containerID="cri-o://fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6" gracePeriod=30
Oct 30 18:48:06.260237 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.260150    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:06.461241 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.461210    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:06.758473 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.758438    4925 generic.go:356] "Generic (PLEG): container finished" podID="dd445b11-79cb-4d2a-a8b5-3b386ea146e8" containerID="4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd" exitCode=0
Oct 30 18:48:06.759154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.758487    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"
Oct 30 18:48:06.759154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.758536    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk" event={"ID":"dd445b11-79cb-4d2a-a8b5-3b386ea146e8","Type":"ContainerDied","Data":"4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd"}
Oct 30 18:48:06.759154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.758591    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk" event={"ID":"dd445b11-79cb-4d2a-a8b5-3b386ea146e8","Type":"ContainerDied","Data":"e260e56b7a43ac77ff7be66e8f0762ee709a19ba0ab087ebaaadea860716ae0e"}
Oct 30 18:48:06.759154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.758622    4925 scope.go:122] "RemoveContainer" containerID="4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd"
Oct 30 18:48:06.769129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.769090    4925 scope.go:122] "RemoveContainer" containerID="4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd"
Oct 30 18:48:06.769428 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:48:06.769404    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd\": container with ID starting with 4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd not found: ID does not exist" containerID="4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd"
Oct 30 18:48:06.769483 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.769438    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd"} err="failed to get container status \"4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd\": rpc error: code = NotFound desc = could not find container \"4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd\": container with ID starting with 4029ede8a4d664a2a63661594dd8dc9341716444a52b5697f4970e4c075ca6dd not found: ID does not exist"
Oct 30 18:48:06.781407 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.781387    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"]
Oct 30 18:48:06.785678 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:06.785657    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-6156d5d6-55454bd86-262hk"]
Oct 30 18:48:07.086331 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:07.086255    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:07.273176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:07.273141    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:07.575270 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:07.575241    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:07.760833 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:07.760803    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:07.991704 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:07.991666    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:08.279159 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:08.279060    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:08.421600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:08.421567    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd445b11-79cb-4d2a-a8b5-3b386ea146e8" path="/var/lib/kubelet/pods/dd445b11-79cb-4d2a-a8b5-3b386ea146e8/volumes"
Oct 30 18:48:08.482070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:08.482030    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:08.673888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:08.673855    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:08.882987 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:08.882948    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:09.530121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:09.529317    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:09.725689 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:09.725650    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:10.086039 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.086002    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"]
Oct 30 18:48:10.086540 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.086249    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg" podUID="cda3d3e1-272d-4b12-9ef3-4a9092809f9e" containerName="maas-api" containerID="cri-o://5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7" gracePeriod=30
Oct 30 18:48:10.258549 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.258494    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:10.359135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.359107    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:48:10.470514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.470480    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"]
Oct 30 18:48:10.470989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.470946    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk" podUID="34f503b7-7b08-4d20-9f68-35a328fb8693" containerName="maas-api" containerID="cri-o://5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b" gracePeriod=30
Oct 30 18:48:10.475284 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.474245    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:10.539641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.539600    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-kube-api-access-hfsqn\" (UniqueName: \"kubernetes.io/projected/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-kube-api-access-hfsqn\") pod \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") "
Oct 30 18:48:10.539778 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.539657    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-maas-api-tls\") pod \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") "
Oct 30 18:48:10.539778 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.539732    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-openshift-service-ca\") pod \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") "
Oct 30 18:48:10.539877 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.539811    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-metrics-certs\") pod \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\" (UID: \"cda3d3e1-272d-4b12-9ef3-4a9092809f9e\") "
Oct 30 18:48:10.540212 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.540189    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-openshift-service-ca" pod "cda3d3e1-272d-4b12-9ef3-4a9092809f9e" (UID: "cda3d3e1-272d-4b12-9ef3-4a9092809f9e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:10.541770 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.541739    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-kube-api-access-hfsqn" pod "cda3d3e1-272d-4b12-9ef3-4a9092809f9e" (UID: "cda3d3e1-272d-4b12-9ef3-4a9092809f9e"). InnerVolumeSpecName "kube-api-access-hfsqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:10.541929 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.541910    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-metrics-certs" pod "cda3d3e1-272d-4b12-9ef3-4a9092809f9e" (UID: "cda3d3e1-272d-4b12-9ef3-4a9092809f9e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:10.542011 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.541985    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-maas-api-tls" pod "cda3d3e1-272d-4b12-9ef3-4a9092809f9e" (UID: "cda3d3e1-272d-4b12-9ef3-4a9092809f9e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:10.627174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.624396    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"]
Oct 30 18:48:10.627174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.624720    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerName="istio-proxy" containerID="cri-o://bfe2ef3d78334eba77ede2c63957a11cd1d5b1174de97036364fd891711498fe" gracePeriod=30
Oct 30 18:48:10.643170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.641088    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:10.643170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.641130    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:10.643170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.641139    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:10.643170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.641148    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfsqn\" (UniqueName: \"kubernetes.io/projected/cda3d3e1-272d-4b12-9ef3-4a9092809f9e-kube-api-access-hfsqn\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:10.751348 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.751307    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:48:10.792511 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.792477    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:10.799177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.797830    4925 generic.go:356] "Generic (PLEG): container finished" podID="34f503b7-7b08-4d20-9f68-35a328fb8693" containerID="5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b" exitCode=0
Oct 30 18:48:10.799177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.797919    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk" event={"ID":"34f503b7-7b08-4d20-9f68-35a328fb8693","Type":"ContainerDied","Data":"5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b"}
Oct 30 18:48:10.799177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.797957    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk" event={"ID":"34f503b7-7b08-4d20-9f68-35a328fb8693","Type":"ContainerDied","Data":"4620ac422b6bdcfecc1a1e6ff0269e4af0377c9bd9c148a61fd4917d8dfcbf3e"}
Oct 30 18:48:10.799177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.797978    4925 scope.go:122] "RemoveContainer" containerID="5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b"
Oct 30 18:48:10.799177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.798075    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"
Oct 30 18:48:10.800200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.800175    4925 generic.go:356] "Generic (PLEG): container finished" podID="cda3d3e1-272d-4b12-9ef3-4a9092809f9e" containerID="5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7" exitCode=0
Oct 30 18:48:10.800319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.800232    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"
Oct 30 18:48:10.800319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.800232    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg" event={"ID":"cda3d3e1-272d-4b12-9ef3-4a9092809f9e","Type":"ContainerDied","Data":"5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7"}
Oct 30 18:48:10.800319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.800311    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg" event={"ID":"cda3d3e1-272d-4b12-9ef3-4a9092809f9e","Type":"ContainerDied","Data":"ae8092fb254953c68cda5ca1a7edbbf5c6ad3bd56709fd7c071e90d50ca0d611"}
Oct 30 18:48:10.810696 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.810682    4925 scope.go:122] "RemoveContainer" containerID="5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b"
Oct 30 18:48:10.811188 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:48:10.811151    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b\": container with ID starting with 5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b not found: ID does not exist" containerID="5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b"
Oct 30 18:48:10.811309 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.811198    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b"} err="failed to get container status \"5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b\": rpc error: code = NotFound desc = could not find container \"5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b\": container with ID starting with 5540662b54e87ed20ff8f5c07ebcb40443d0279ede9dbda61c53300d7e8e793b not found: ID does not exist"
Oct 30 18:48:10.811309 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.811220    4925 scope.go:122] "RemoveContainer" containerID="5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7"
Oct 30 18:48:10.821275 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.821257    4925 scope.go:122] "RemoveContainer" containerID="5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7"
Oct 30 18:48:10.821553 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:48:10.821525    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7\": container with ID starting with 5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7 not found: ID does not exist" containerID="5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7"
Oct 30 18:48:10.821644 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.821559    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7"} err="failed to get container status \"5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7\": rpc error: code = NotFound desc = could not find container \"5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7\": container with ID starting with 5419fc2c0cc1c14be54e97e2582c2c843a823ac0ed1e664cd3a28aa4655b6ca7 not found: ID does not exist"
Oct 30 18:48:10.827870 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.827843    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"]
Oct 30 18:48:10.830703 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.830682    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-226ecbcb-5d9b956645-gj9jg"]
Oct 30 18:48:10.843179 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.843161    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-maas-api-tls\") pod \"34f503b7-7b08-4d20-9f68-35a328fb8693\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") "
Oct 30 18:48:10.843294 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.843193    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-metrics-certs\" (UniqueName: \"kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-metrics-certs\") pod \"34f503b7-7b08-4d20-9f68-35a328fb8693\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") "
Oct 30 18:48:10.843294 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.843231    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34f503b7-7b08-4d20-9f68-35a328fb8693-kube-api-access-6nhfn\" (UniqueName: \"kubernetes.io/projected/34f503b7-7b08-4d20-9f68-35a328fb8693-kube-api-access-6nhfn\") pod \"34f503b7-7b08-4d20-9f68-35a328fb8693\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") "
Oct 30 18:48:10.843294 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.843250    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34f503b7-7b08-4d20-9f68-35a328fb8693-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34f503b7-7b08-4d20-9f68-35a328fb8693-openshift-service-ca\") pod \"34f503b7-7b08-4d20-9f68-35a328fb8693\" (UID: \"34f503b7-7b08-4d20-9f68-35a328fb8693\") "
Oct 30 18:48:10.843655 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.843632    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34f503b7-7b08-4d20-9f68-35a328fb8693-openshift-service-ca" pod "34f503b7-7b08-4d20-9f68-35a328fb8693" (UID: "34f503b7-7b08-4d20-9f68-35a328fb8693"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:10.845416 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.845392    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-maas-api-tls" pod "34f503b7-7b08-4d20-9f68-35a328fb8693" (UID: "34f503b7-7b08-4d20-9f68-35a328fb8693"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:10.845416 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.845413    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-metrics-certs" pod "34f503b7-7b08-4d20-9f68-35a328fb8693" (UID: "34f503b7-7b08-4d20-9f68-35a328fb8693"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:10.845528 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.845431    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34f503b7-7b08-4d20-9f68-35a328fb8693-kube-api-access-6nhfn" pod "34f503b7-7b08-4d20-9f68-35a328fb8693" (UID: "34f503b7-7b08-4d20-9f68-35a328fb8693"). InnerVolumeSpecName "kube-api-access-6nhfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:10.944212 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.944183    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:10.944212 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.944212    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34f503b7-7b08-4d20-9f68-35a328fb8693-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:10.944512 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.944222    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nhfn\" (UniqueName: \"kubernetes.io/projected/34f503b7-7b08-4d20-9f68-35a328fb8693-kube-api-access-6nhfn\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:10.944512 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.944232    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34f503b7-7b08-4d20-9f68-35a328fb8693-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:10.985287 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:10.985253    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:11.146592 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.146537    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"]
Oct 30 18:48:11.155864 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.155836    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ee87bf29-7c649868f4-zrnrk"]
Oct 30 18:48:11.193990 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.193960    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:11.266121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.264334    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"]
Oct 30 18:48:11.396042 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.396015    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:11.455211 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.455157    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:48:11.550504 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.550470    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-data\") pod \"703c1835-5274-40b5-a967-096296705e15\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") "
Oct 30 18:48:11.550696 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.550537    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-certs\") pod \"703c1835-5274-40b5-a967-096296705e15\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") "
Oct 30 18:48:11.550696 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.550566    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-socket\") pod \"703c1835-5274-40b5-a967-096296705e15\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") "
Oct 30 18:48:11.550696 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.550600    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-kube-api-access-fwx59\" (UniqueName: \"kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-kube-api-access-fwx59\") pod \"703c1835-5274-40b5-a967-096296705e15\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") "
Oct 30 18:48:11.550696 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.550665    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-envoy\") pod \"703c1835-5274-40b5-a967-096296705e15\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") "
Oct 30 18:48:11.550696 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.550690    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/703c1835-5274-40b5-a967-096296705e15-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/703c1835-5274-40b5-a967-096296705e15-istiod-ca-cert\") pod \"703c1835-5274-40b5-a967-096296705e15\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") "
Oct 30 18:48:11.550947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.550776    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-data" pod "703c1835-5274-40b5-a967-096296705e15" (UID: "703c1835-5274-40b5-a967-096296705e15"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:11.550947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.550854    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-socket" pod "703c1835-5274-40b5-a967-096296705e15" (UID: "703c1835-5274-40b5-a967-096296705e15"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:11.551018 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.550965    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-certs" pod "703c1835-5274-40b5-a967-096296705e15" (UID: "703c1835-5274-40b5-a967-096296705e15"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:11.551088 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.551071    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:11.551146 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.551111    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:11.551146 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.551127    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:11.551214 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.551185    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/703c1835-5274-40b5-a967-096296705e15-istiod-ca-cert" pod "703c1835-5274-40b5-a967-096296705e15" (UID: "703c1835-5274-40b5-a967-096296705e15"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:11.553221 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.553194    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-kube-api-access-fwx59" pod "703c1835-5274-40b5-a967-096296705e15" (UID: "703c1835-5274-40b5-a967-096296705e15"). InnerVolumeSpecName "kube-api-access-fwx59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:11.553335 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.553282    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-envoy" pod "703c1835-5274-40b5-a967-096296705e15" (UID: "703c1835-5274-40b5-a967-096296705e15"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:11.594268 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.594234    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:11.651743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.651716    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/703c1835-5274-40b5-a967-096296705e15-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/703c1835-5274-40b5-a967-096296705e15-istio-podinfo\") pod \"703c1835-5274-40b5-a967-096296705e15\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") "
Oct 30 18:48:11.651872 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.651754    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-istio-token\" (UniqueName: \"kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-istio-token\") pod \"703c1835-5274-40b5-a967-096296705e15\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") "
Oct 30 18:48:11.651872 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.651807    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-credential-socket\") pod \"703c1835-5274-40b5-a967-096296705e15\" (UID: \"703c1835-5274-40b5-a967-096296705e15\") "
Oct 30 18:48:11.652077 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.652055    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-credential-socket" pod "703c1835-5274-40b5-a967-096296705e15" (UID: "703c1835-5274-40b5-a967-096296705e15"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:11.652235 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.652217    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwx59\" (UniqueName: \"kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-kube-api-access-fwx59\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:11.652280 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.652245    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:11.652280 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.652262    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/703c1835-5274-40b5-a967-096296705e15-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:11.652280 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.652276    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/703c1835-5274-40b5-a967-096296705e15-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:11.653881 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.653860    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-istio-token" pod "703c1835-5274-40b5-a967-096296705e15" (UID: "703c1835-5274-40b5-a967-096296705e15"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:11.653881 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.653874    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/703c1835-5274-40b5-a967-096296705e15-istio-podinfo" pod "703c1835-5274-40b5-a967-096296705e15" (UID: "703c1835-5274-40b5-a967-096296705e15"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:48:11.753743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.753663    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/703c1835-5274-40b5-a967-096296705e15-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:11.753743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.753692    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/703c1835-5274-40b5-a967-096296705e15-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:11.809223 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.809191    4925 generic.go:356] "Generic (PLEG): container finished" podID="703c1835-5274-40b5-a967-096296705e15" containerID="fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6" exitCode=0
Oct 30 18:48:11.809379 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.809248    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"
Oct 30 18:48:11.809379 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.809256    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" event={"ID":"703c1835-5274-40b5-a967-096296705e15","Type":"ContainerDied","Data":"fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6"}
Oct 30 18:48:11.809379 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.809300    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s" event={"ID":"703c1835-5274-40b5-a967-096296705e15","Type":"ContainerDied","Data":"4c89d3949423ad5266c0c46c462b22ee3af8447aec8c1004bda0866a8780d8e9"}
Oct 30 18:48:11.809379 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.809323    4925 scope.go:122] "RemoveContainer" containerID="fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6"
Oct 30 18:48:11.819783 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.819764    4925 scope.go:122] "RemoveContainer" containerID="baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c"
Oct 30 18:48:11.828888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.828870    4925 scope.go:122] "RemoveContainer" containerID="fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6"
Oct 30 18:48:11.829201 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:48:11.829182    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6\": container with ID starting with fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6 not found: ID does not exist" containerID="fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6"
Oct 30 18:48:11.829261 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.829210    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6"} err="failed to get container status \"fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6\": rpc error: code = NotFound desc = could not find container \"fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6\": container with ID starting with fb9e71410c5cce8185deaca3897cd5f6fbec915d396dd2fb9b4a7f7d71a397c6 not found: ID does not exist"
Oct 30 18:48:11.829261 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.829225    4925 scope.go:122] "RemoveContainer" containerID="baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c"
Oct 30 18:48:11.829458 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:48:11.829437    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c\": container with ID starting with baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c not found: ID does not exist" containerID="baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c"
Oct 30 18:48:11.829496 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.829466    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c"} err="failed to get container status \"baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c\": rpc error: code = NotFound desc = could not find container \"baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c\": container with ID starting with baf1b68dd433fcc4265efee5f80b340dc561826b3c414d6ece51a96cc8e88b7c not found: ID does not exist"
Oct 30 18:48:11.839262 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.839236    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"]
Oct 30 18:48:11.856003 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.855982    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-6156d5d6-gw-openshift-default-7469cb758f-v924s"]
Oct 30 18:48:11.901121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.900288    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:11.901121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.900369    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"]
Oct 30 18:48:11.901121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901012    4925 cpu_manager.go:414] "containerMap: removing container" podUID="cda3d3e1-272d-4b12-9ef3-4a9092809f9e" containerName="maas-api"
Oct 30 18:48:11.901121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901032    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cda3d3e1-272d-4b12-9ef3-4a9092809f9e" containerName="maas-api"
Oct 30 18:48:11.901121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901045    4925 cpu_manager.go:414] "containerMap: removing container" podUID="34f503b7-7b08-4d20-9f68-35a328fb8693" containerName="maas-api"
Oct 30 18:48:11.901121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901051    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34f503b7-7b08-4d20-9f68-35a328fb8693" containerName="maas-api"
Oct 30 18:48:11.901121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901083    4925 cpu_manager.go:414] "containerMap: removing container" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy"
Oct 30 18:48:11.901121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901110    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy"
Oct 30 18:48:11.901121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901121    4925 cpu_manager.go:414] "containerMap: removing container" podUID="dd445b11-79cb-4d2a-a8b5-3b386ea146e8" containerName="maas-api"
Oct 30 18:48:11.901121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901127    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd445b11-79cb-4d2a-a8b5-3b386ea146e8" containerName="maas-api"
Oct 30 18:48:11.901727 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901139    4925 cpu_manager.go:414] "containerMap: removing container" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy"
Oct 30 18:48:11.901727 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901146    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy"
Oct 30 18:48:11.901727 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901259    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy"
Oct 30 18:48:11.901727 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901271    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd445b11-79cb-4d2a-a8b5-3b386ea146e8" containerName="maas-api"
Oct 30 18:48:11.901727 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901279    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="703c1835-5274-40b5-a967-096296705e15" containerName="istio-proxy"
Oct 30 18:48:11.901727 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901291    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="cda3d3e1-272d-4b12-9ef3-4a9092809f9e" containerName="maas-api"
Oct 30 18:48:11.901727 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.901297    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="34f503b7-7b08-4d20-9f68-35a328fb8693" containerName="maas-api"
Oct 30 18:48:11.907158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.907138    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:11.918489 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.918465    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-39bde3-openshift-default-dockercfg-vwm86\""
Oct 30 18:48:11.924340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.922054    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"]
Oct 30 18:48:11.956047 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.956020    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:11.956196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.956053    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:11.956196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.956083    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:11.956318 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.956210    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:11.956370 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.956330    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:11.956370 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.956357    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:11.956472 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.956458    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:11.956525 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.956512    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49765651-6ef5-47c8-aef6-03b21952a2ad-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:11.956576 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:11.956550    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg9kr\" (UniqueName: \"kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-kube-api-access-mg9kr\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057029    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49765651-6ef5-47c8-aef6-03b21952a2ad-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057074    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg9kr\" (UniqueName: \"kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-kube-api-access-mg9kr\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057156    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057182    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057211    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057253    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057304    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057328    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057637 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057384    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057637 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057579    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057742 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057636    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057823 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057801    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.057999 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.057974    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.058372 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.058350    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49765651-6ef5-47c8-aef6-03b21952a2ad-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.060654 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.060629    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.062201 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.062180    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.067351 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.067324    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg9kr\" (UniqueName: \"kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-kube-api-access-mg9kr\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.067840 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.067820    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.123526 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.123479    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:12.225481 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.225445    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:12.401254 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.394961    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"]
Oct 30 18:48:12.401254 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.395196    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy" containerID="cri-o://d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329" gracePeriod=30
Oct 30 18:48:12.422781 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.422755    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34f503b7-7b08-4d20-9f68-35a328fb8693" path="/var/lib/kubelet/pods/34f503b7-7b08-4d20-9f68-35a328fb8693/volumes"
Oct 30 18:48:12.423235 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.423219    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="703c1835-5274-40b5-a967-096296705e15" path="/var/lib/kubelet/pods/703c1835-5274-40b5-a967-096296705e15/volumes"
Oct 30 18:48:12.423645 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.423631    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cda3d3e1-272d-4b12-9ef3-4a9092809f9e" path="/var/lib/kubelet/pods/cda3d3e1-272d-4b12-9ef3-4a9092809f9e/volumes"
Oct 30 18:48:12.525117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.524754    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"]
Oct 30 18:48:12.540597 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.540553    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:48:12.540719 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.540700    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:48:12.540823 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.540789    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:48:12.610130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.610084    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"]
Oct 30 18:48:12.613660 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.613641    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"
Oct 30 18:48:12.617567 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.617551    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-2889dd\""
Oct 30 18:48:12.617767 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.617752    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-2889dd-dockercfg-nxnzl\""
Oct 30 18:48:12.628318 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.628296    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"]
Oct 30 18:48:12.650030 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.649997    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"]
Oct 30 18:48:12.654681 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.654608    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"
Oct 30 18:48:12.685117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.678355    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"]
Oct 30 18:48:12.685117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.683964    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ae0a880-733c-4773-ad10-7239533263df-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz\" (UID: \"9ae0a880-733c-4773-ad10-7239533263df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"
Oct 30 18:48:12.685117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.684035    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wstcb\" (UniqueName: \"kubernetes.io/projected/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-kube-api-access-wstcb\") pod \"payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4\" (UID: \"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"
Oct 30 18:48:12.685117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.684155    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4\" (UID: \"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"
Oct 30 18:48:12.685117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.684200    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65flt\" (UniqueName: \"kubernetes.io/projected/9ae0a880-733c-4773-ad10-7239533263df-kube-api-access-65flt\") pod \"payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz\" (UID: \"9ae0a880-733c-4773-ad10-7239533263df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"
Oct 30 18:48:12.785473 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.785425    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4\" (UID: \"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"
Oct 30 18:48:12.785660 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.785492    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65flt\" (UniqueName: \"kubernetes.io/projected/9ae0a880-733c-4773-ad10-7239533263df-kube-api-access-65flt\") pod \"payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz\" (UID: \"9ae0a880-733c-4773-ad10-7239533263df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"
Oct 30 18:48:12.785660 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.785564    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ae0a880-733c-4773-ad10-7239533263df-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz\" (UID: \"9ae0a880-733c-4773-ad10-7239533263df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"
Oct 30 18:48:12.785660 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.785626    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wstcb\" (UniqueName: \"kubernetes.io/projected/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-kube-api-access-wstcb\") pod \"payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4\" (UID: \"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"
Oct 30 18:48:12.786883 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.786852    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ae0a880-733c-4773-ad10-7239533263df-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz\" (UID: \"9ae0a880-733c-4773-ad10-7239533263df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"
Oct 30 18:48:12.786883 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.786869    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4\" (UID: \"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"
Oct 30 18:48:12.800013 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.799891    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65flt\" (UniqueName: \"kubernetes.io/projected/9ae0a880-733c-4773-ad10-7239533263df-kube-api-access-65flt\") pod \"payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz\" (UID: \"9ae0a880-733c-4773-ad10-7239533263df\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"
Oct 30 18:48:12.801383 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.801356    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wstcb\" (UniqueName: \"kubernetes.io/projected/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-kube-api-access-wstcb\") pod \"payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4\" (UID: \"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"
Oct 30 18:48:12.816745 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.816710    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" event={"ID":"49765651-6ef5-47c8-aef6-03b21952a2ad","Type":"ContainerStarted","Data":"11512d0d37ba8bc25d3d68a508f18025278b9b8f2c345a3462ce9178403bb53d"}
Oct 30 18:48:12.816865 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.816778    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" event={"ID":"49765651-6ef5-47c8-aef6-03b21952a2ad","Type":"ContainerStarted","Data":"dc7e53e742c632f073f15a30ba34db8d053ed3f90a5ccc5c6334dc509904d6e7"}
Oct 30 18:48:12.838772 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.838730    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" podStartSLOduration=1.8387172779999998 podStartE2EDuration="1.838717278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:12.836529577 +0000 UTC m=+714.552874654" watchObservedRunningTime="2026-10-30 18:48:12.838717278 +0000 UTC m=+714.555062354"
Oct 30 18:48:12.927419 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.927390    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"
Oct 30 18:48:12.992320 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:12.992288    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"
Oct 30 18:48:13.029076 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.029048    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"]
Oct 30 18:48:13.034060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.033608    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.051038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.050796    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"]
Oct 30 18:48:13.089705 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.089673    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-data\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.089828 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.089738    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.089828 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.089797    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-envoy\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.089949 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.089835    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368eed20-8aa8-4387-a513-6be2459125dc-istiod-ca-cert\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.089949 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.089882    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368eed20-8aa8-4387-a513-6be2459125dc-istio-podinfo\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.089949 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.089930    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-istio-token\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.090133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.089961    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-certs\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.090133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.090003    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4qt6\" (UniqueName: \"kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-kube-api-access-c4qt6\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.090248 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.090147    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-credential-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.135149 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.131765    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-9de872-openshift-default-dockercfg-9bf8l\""
Oct 30 18:48:13.191144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.190924    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-istio-token\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.191744 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.191715    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-certs\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.192347 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.192327    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4qt6\" (UniqueName: \"kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-kube-api-access-c4qt6\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.192793 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.192771    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-credential-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.193054 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.193033    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-data\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.193256 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.193238    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.193703 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.193683    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-envoy\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.193888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.193869    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368eed20-8aa8-4387-a513-6be2459125dc-istiod-ca-cert\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.194045 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.194027    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368eed20-8aa8-4387-a513-6be2459125dc-istio-podinfo\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.194215 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.192278    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-certs\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.194371 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.193607    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.194470 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.193133    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-credential-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.194558 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.193386    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-data\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.194968 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.194938    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368eed20-8aa8-4387-a513-6be2459125dc-istiod-ca-cert\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.202383 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.202331    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-envoy\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.203337 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.203308    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-istio-token\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.206442 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.206418    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4qt6\" (UniqueName: \"kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-kube-api-access-c4qt6\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.209922 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.209855    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368eed20-8aa8-4387-a513-6be2459125dc-istio-podinfo\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.226323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.226209    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:13.226722 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.226330    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"]
Oct 30 18:48:13.227316 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:48:13.227293    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ae0a880_733c_4773_ad10_7239533263df.slice/crio-6eea16254cd2daf9f3b90871269ffbef5bd97efdae449291ce20731ebf89552c WatchSource:0}: Error finding container 6eea16254cd2daf9f3b90871269ffbef5bd97efdae449291ce20731ebf89552c: Status 404 returned error can't find the container with id 6eea16254cd2daf9f3b90871269ffbef5bd97efdae449291ce20731ebf89552c
Oct 30 18:48:13.232661 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.232642    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:13.306424 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:48:13.306395    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:48:13.311933 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.311906    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"]
Oct 30 18:48:13.313720 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:48:13.313693    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1670ef9a_3a4d_43ca_b0fd_4ca6e6735b07.slice/crio-77259d9984b9500f31e2b633b333fbe74a3e2edb0a4fb4a45afcdf21d4b42a57 WatchSource:0}: Error finding container 77259d9984b9500f31e2b633b333fbe74a3e2edb0a4fb4a45afcdf21d4b42a57: Status 404 returned error can't find the container with id 77259d9984b9500f31e2b633b333fbe74a3e2edb0a4fb4a45afcdf21d4b42a57
Oct 30 18:48:13.358874 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.358846    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:13.526433 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.526350    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:13.678299 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.678265    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"]
Oct 30 18:48:13.686685 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:48:13.686657    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod368eed20_8aa8_4387_a513_6be2459125dc.slice/crio-0835f057a6b8d13ab3067f0625bbd300f4482a6c1155a1566fd3f01bff70ac15 WatchSource:0}: Error finding container 0835f057a6b8d13ab3067f0625bbd300f4482a6c1155a1566fd3f01bff70ac15: Status 404 returned error can't find the container with id 0835f057a6b8d13ab3067f0625bbd300f4482a6c1155a1566fd3f01bff70ac15
Oct 30 18:48:13.689875 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.689843    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:48:13.689981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.689915    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:48:13.689981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.689945    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:48:13.836893 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.836816    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" event={"ID":"368eed20-8aa8-4387-a513-6be2459125dc","Type":"ContainerStarted","Data":"37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714"}
Oct 30 18:48:13.837396 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.837368    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" event={"ID":"368eed20-8aa8-4387-a513-6be2459125dc","Type":"ContainerStarted","Data":"0835f057a6b8d13ab3067f0625bbd300f4482a6c1155a1566fd3f01bff70ac15"}
Oct 30 18:48:13.838923 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.838863    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4" event={"ID":"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07","Type":"ContainerStarted","Data":"3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15"}
Oct 30 18:48:13.838923 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.838904    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4" event={"ID":"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07","Type":"ContainerStarted","Data":"77259d9984b9500f31e2b633b333fbe74a3e2edb0a4fb4a45afcdf21d4b42a57"}
Oct 30 18:48:13.839128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.838934    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"
Oct 30 18:48:13.841575 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.841432    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz" event={"ID":"9ae0a880-733c-4773-ad10-7239533263df","Type":"ContainerStarted","Data":"972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f"}
Oct 30 18:48:13.841575 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.841471    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz" event={"ID":"9ae0a880-733c-4773-ad10-7239533263df","Type":"ContainerStarted","Data":"6eea16254cd2daf9f3b90871269ffbef5bd97efdae449291ce20731ebf89552c"}
Oct 30 18:48:13.841575 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.841500    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"
Oct 30 18:48:13.841575 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.841522    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:13.843007 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.842963    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:13.902993 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.902942    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" podStartSLOduration=0.902928118 podStartE2EDuration="902.928118ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:13.901726334 +0000 UTC m=+715.618071411" watchObservedRunningTime="2026-10-30 18:48:13.902928118 +0000 UTC m=+715.619273194"
Oct 30 18:48:13.971267 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:13.971231    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:14.010690 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:14.010614    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz" podStartSLOduration=2.010590129 podStartE2EDuration="2.010590129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:14.010024755 +0000 UTC m=+715.726369832" watchObservedRunningTime="2026-10-30 18:48:14.010590129 +0000 UTC m=+715.726935291"
Oct 30 18:48:14.010888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:14.010734    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4" podStartSLOduration=2.010727518 podStartE2EDuration="2.010727518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:13.968194603 +0000 UTC m=+715.684539680" watchObservedRunningTime="2026-10-30 18:48:14.010727518 +0000 UTC m=+715.727072594"
Oct 30 18:48:14.175230 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:14.175197    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:14.359703 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:14.359664    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:14.367238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:14.366594    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:14.550927 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:14.550843    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:14.846822 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:14.846746    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:14.847728 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:14.847702    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:15.482875 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:15.482834    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:15.719076 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:15.719046    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:15.859348 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:15.859081    4925 generic.go:356] "Generic (PLEG): container finished" podID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerID="bfe2ef3d78334eba77ede2c63957a11cd1d5b1174de97036364fd891711498fe" exitCode=0
Oct 30 18:48:15.859348 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:15.859230    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" event={"ID":"56fbb51c-5be2-4b4e-90eb-fe3e81060f80","Type":"ContainerDied","Data":"bfe2ef3d78334eba77ede2c63957a11cd1d5b1174de97036364fd891711498fe"}
Oct 30 18:48:15.859348 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:15.859298    4925 scope.go:122] "RemoveContainer" containerID="e6545be4eed74c3772676c28b35a21db0be7a39eaecbb8d395273fd37a1a779c"
Oct 30 18:48:15.927699 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:15.927674    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:48:16.028074 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.027995    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-credential-socket\") pod \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") "
Oct 30 18:48:16.028074 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028050    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-data\") pod \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") "
Oct 30 18:48:16.028315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028076    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-kube-api-access-9r5lz\" (UniqueName: \"kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-kube-api-access-9r5lz\") pod \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") "
Oct 30 18:48:16.028315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028112    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istiod-ca-cert\") pod \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") "
Oct 30 18:48:16.028315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028163    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-envoy\") pod \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") "
Oct 30 18:48:16.028315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028206    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-certs\") pod \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") "
Oct 30 18:48:16.028315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028235    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-podinfo\") pod \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") "
Oct 30 18:48:16.028315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028296    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-token\" (UniqueName: \"kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-token\") pod \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") "
Oct 30 18:48:16.028628 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028324    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-credential-socket" pod "56fbb51c-5be2-4b4e-90eb-fe3e81060f80" (UID: "56fbb51c-5be2-4b4e-90eb-fe3e81060f80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:16.028628 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028333    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-socket\") pod \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\" (UID: \"56fbb51c-5be2-4b4e-90eb-fe3e81060f80\") "
Oct 30 18:48:16.028628 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028377    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-data" pod "56fbb51c-5be2-4b4e-90eb-fe3e81060f80" (UID: "56fbb51c-5be2-4b4e-90eb-fe3e81060f80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:16.028628 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028545    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-certs" pod "56fbb51c-5be2-4b4e-90eb-fe3e81060f80" (UID: "56fbb51c-5be2-4b4e-90eb-fe3e81060f80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:16.028628 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028568    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istiod-ca-cert" pod "56fbb51c-5be2-4b4e-90eb-fe3e81060f80" (UID: "56fbb51c-5be2-4b4e-90eb-fe3e81060f80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:16.028903 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028805    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-socket" pod "56fbb51c-5be2-4b4e-90eb-fe3e81060f80" (UID: "56fbb51c-5be2-4b4e-90eb-fe3e81060f80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:16.028903 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028877    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:16.028903 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028900    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:16.029025 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.028944    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:16.029141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.029122    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:16.030734 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.030705    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-envoy" pod "56fbb51c-5be2-4b4e-90eb-fe3e81060f80" (UID: "56fbb51c-5be2-4b4e-90eb-fe3e81060f80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:16.030895 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.030875    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-podinfo" pod "56fbb51c-5be2-4b4e-90eb-fe3e81060f80" (UID: "56fbb51c-5be2-4b4e-90eb-fe3e81060f80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:48:16.031039 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.031020    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-kube-api-access-9r5lz" pod "56fbb51c-5be2-4b4e-90eb-fe3e81060f80" (UID: "56fbb51c-5be2-4b4e-90eb-fe3e81060f80"). InnerVolumeSpecName "kube-api-access-9r5lz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:16.031082 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.031049    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-token" pod "56fbb51c-5be2-4b4e-90eb-fe3e81060f80" (UID: "56fbb51c-5be2-4b4e-90eb-fe3e81060f80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:16.130611 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.130477    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:16.130611 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.130513    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:16.130611 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.130528    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:16.130611 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.130543    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r5lz\" (UniqueName: \"kubernetes.io/projected/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-kube-api-access-9r5lz\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:16.130611 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.130575    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56fbb51c-5be2-4b4e-90eb-fe3e81060f80-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:16.265177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.264925    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2" podUID="9f3f5c7e-f689-4f25-9909-7766ebe2ff1e" containerName="payload-pre-processing" containerID="cri-o://5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45" gracePeriod=25
Oct 30 18:48:16.464048 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.464013    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:16.566631 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.565514    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"
Oct 30 18:48:16.638348 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.638278    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-plugins-config-volume\") pod \"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e\" (UID: \"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e\") "
Oct 30 18:48:16.638481 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.638399    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-kube-api-access-qztz5\" (UniqueName: \"kubernetes.io/projected/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-kube-api-access-qztz5\") pod \"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e\" (UID: \"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e\") "
Oct 30 18:48:16.638670 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.638640    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-plugins-config-volume" pod "9f3f5c7e-f689-4f25-9909-7766ebe2ff1e" (UID: "9f3f5c7e-f689-4f25-9909-7766ebe2ff1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:16.640556 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.640535    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-kube-api-access-qztz5" pod "9f3f5c7e-f689-4f25-9909-7766ebe2ff1e" (UID: "9f3f5c7e-f689-4f25-9909-7766ebe2ff1e"). InnerVolumeSpecName "kube-api-access-qztz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:16.667935 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.667911    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:16.739579 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.739545    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qztz5\" (UniqueName: \"kubernetes.io/projected/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-kube-api-access-qztz5\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:16.739579 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.739572    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:16.845618 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.845583    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"]
Oct 30 18:48:16.846012 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.845999    4925 cpu_manager.go:414] "containerMap: removing container" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerName="istio-proxy"
Oct 30 18:48:16.846056 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.846013    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerName="istio-proxy"
Oct 30 18:48:16.846056 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.846046    4925 cpu_manager.go:414] "containerMap: removing container" podUID="9f3f5c7e-f689-4f25-9909-7766ebe2ff1e" containerName="payload-pre-processing"
Oct 30 18:48:16.846056 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.846051    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f3f5c7e-f689-4f25-9909-7766ebe2ff1e" containerName="payload-pre-processing"
Oct 30 18:48:16.846193 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.846126    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerName="istio-proxy"
Oct 30 18:48:16.846193 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.846139    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f3f5c7e-f689-4f25-9909-7766ebe2ff1e" containerName="payload-pre-processing"
Oct 30 18:48:16.849349 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.849331    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 18:48:16.860522 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.860492    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-39bde3\""
Oct 30 18:48:16.860716 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.860698    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-39bde3-dockercfg-hq85c\""
Oct 30 18:48:16.865580 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.865558    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"]
Oct 30 18:48:16.866854 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.866835    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"
Oct 30 18:48:16.866962 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.866873    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q" event={"ID":"56fbb51c-5be2-4b4e-90eb-fe3e81060f80","Type":"ContainerDied","Data":"4fc26ef3c5f40999edab5ff962fba1941f54c9419ae543e73a233cf32f48b075"}
Oct 30 18:48:16.866962 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.866921    4925 scope.go:122] "RemoveContainer" containerID="bfe2ef3d78334eba77ede2c63957a11cd1d5b1174de97036364fd891711498fe"
Oct 30 18:48:16.868563 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.868537    4925 generic.go:356] "Generic (PLEG): container finished" podID="9f3f5c7e-f689-4f25-9909-7766ebe2ff1e" containerID="5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45" exitCode=0
Oct 30 18:48:16.868790 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.868588    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"
Oct 30 18:48:16.869072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.868601    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2" event={"ID":"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e","Type":"ContainerDied","Data":"5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45"}
Oct 30 18:48:16.869319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.869180    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2" event={"ID":"9f3f5c7e-f689-4f25-9909-7766ebe2ff1e","Type":"ContainerDied","Data":"a922d8d37caff377cbcfa46aa706252f4fbdf022e68a13d2166621e49fcfc901"}
Oct 30 18:48:16.879089 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.879063    4925 scope.go:122] "RemoveContainer" containerID="5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45"
Oct 30 18:48:16.887596 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.887577    4925 scope.go:122] "RemoveContainer" containerID="5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45"
Oct 30 18:48:16.887858 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:48:16.887840    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45\": container with ID starting with 5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45 not found: ID does not exist" containerID="5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45"
Oct 30 18:48:16.887906 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.887867    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45"} err="failed to get container status \"5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45\": rpc error: code = NotFound desc = could not find container \"5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45\": container with ID starting with 5b28777c57d6513cf612c23d125b5df237ad34ccf5f8f2b060c6ddb6c9ddcd45 not found: ID does not exist"
Oct 30 18:48:16.915706 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.915686    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"]
Oct 30 18:48:16.931471 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.931449    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-226ecbcb-openshift-default-76b598d4f5-rw84q"]
Oct 30 18:48:16.937371 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.937347    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:16.941591 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.941571    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fkls\" (UniqueName: \"kubernetes.io/projected/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-kube-api-access-7fkls\") pod \"payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws\" (UID: \"84ce6a41-5d27-44aa-86bd-b589a00d0c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 18:48:16.941681 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.941645    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws\" (UID: \"84ce6a41-5d27-44aa-86bd-b589a00d0c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 18:48:16.972062 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.972028    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"]
Oct 30 18:48:16.974048 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:16.974022    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-f951dfa9-64fb47cdc-pffp2"]
Oct 30 18:48:17.043208 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.043171    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fkls\" (UniqueName: \"kubernetes.io/projected/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-kube-api-access-7fkls\") pod \"payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws\" (UID: \"84ce6a41-5d27-44aa-86bd-b589a00d0c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 18:48:17.043349 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.043225    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws\" (UID: \"84ce6a41-5d27-44aa-86bd-b589a00d0c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 18:48:17.044009 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.043978    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws\" (UID: \"84ce6a41-5d27-44aa-86bd-b589a00d0c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 18:48:17.062681 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.062652    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fkls\" (UniqueName: \"kubernetes.io/projected/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-kube-api-access-7fkls\") pod \"payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws\" (UID: \"84ce6a41-5d27-44aa-86bd-b589a00d0c2a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 18:48:17.163293 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.163261    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 18:48:17.184992 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.184962    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:17.443152 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.443127    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"]
Oct 30 18:48:17.491063 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:48:17.491027    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ce6a41_5d27_44aa_86bd_b589a00d0c2a.slice/crio-955b99816732d99434447e17cd336a9e768020f939d9528762aeb78a95f6cc54 WatchSource:0}: Error finding container 955b99816732d99434447e17cd336a9e768020f939d9528762aeb78a95f6cc54: Status 404 returned error can't find the container with id 955b99816732d99434447e17cd336a9e768020f939d9528762aeb78a95f6cc54
Oct 30 18:48:17.680519 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.680306    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:48:17.749964 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.749918    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"]
Oct 30 18:48:17.750164 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750140    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-istio-token\" (UniqueName: \"kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-istio-token\") pod \"c4b48776-3a2c-423c-b695-89de08958a13\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") "
Oct 30 18:48:17.750224 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750189    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-kube-api-access-wp4xg\" (UniqueName: \"kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-kube-api-access-wp4xg\") pod \"c4b48776-3a2c-423c-b695-89de08958a13\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") "
Oct 30 18:48:17.750297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750280    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-envoy\") pod \"c4b48776-3a2c-423c-b695-89de08958a13\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") "
Oct 30 18:48:17.750371 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750355    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4b48776-3a2c-423c-b695-89de08958a13-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b48776-3a2c-423c-b695-89de08958a13-istiod-ca-cert\") pod \"c4b48776-3a2c-423c-b695-89de08958a13\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") "
Oct 30 18:48:17.750424 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750416    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-data\") pod \"c4b48776-3a2c-423c-b695-89de08958a13\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") "
Oct 30 18:48:17.750467 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750446    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-socket\") pod \"c4b48776-3a2c-423c-b695-89de08958a13\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") "
Oct 30 18:48:17.750516 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750475    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-credential-socket\") pod \"c4b48776-3a2c-423c-b695-89de08958a13\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") "
Oct 30 18:48:17.750573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750516    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-certs\") pod \"c4b48776-3a2c-423c-b695-89de08958a13\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") "
Oct 30 18:48:17.750573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750566    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c4b48776-3a2c-423c-b695-89de08958a13-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b48776-3a2c-423c-b695-89de08958a13-istio-podinfo\") pod \"c4b48776-3a2c-423c-b695-89de08958a13\" (UID: \"c4b48776-3a2c-423c-b695-89de08958a13\") "
Oct 30 18:48:17.750719 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750659    4925 cpu_manager.go:414] "containerMap: removing container" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy"
Oct 30 18:48:17.750719 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750678    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy"
Oct 30 18:48:17.750822 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750737    4925 cpu_manager.go:414] "containerMap: removing container" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerName="istio-proxy"
Oct 30 18:48:17.750822 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750746    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerName="istio-proxy"
Oct 30 18:48:17.750914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750855    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy"
Oct 30 18:48:17.750914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.750869    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" containerName="istio-proxy"
Oct 30 18:48:17.751284 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.751254    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-certs" pod "c4b48776-3a2c-423c-b695-89de08958a13" (UID: "c4b48776-3a2c-423c-b695-89de08958a13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:17.751456 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.751390    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-data" pod "c4b48776-3a2c-423c-b695-89de08958a13" (UID: "c4b48776-3a2c-423c-b695-89de08958a13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:17.751587 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.751487    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4b48776-3a2c-423c-b695-89de08958a13-istiod-ca-cert" pod "c4b48776-3a2c-423c-b695-89de08958a13" (UID: "c4b48776-3a2c-423c-b695-89de08958a13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:17.751686 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.751662    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-credential-socket" pod "c4b48776-3a2c-423c-b695-89de08958a13" (UID: "c4b48776-3a2c-423c-b695-89de08958a13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:17.751866 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.751842    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-socket" pod "c4b48776-3a2c-423c-b695-89de08958a13" (UID: "c4b48776-3a2c-423c-b695-89de08958a13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:17.754385 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.754356    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4b48776-3a2c-423c-b695-89de08958a13-istio-podinfo" pod "c4b48776-3a2c-423c-b695-89de08958a13" (UID: "c4b48776-3a2c-423c-b695-89de08958a13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:48:17.754995 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.754967    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-envoy" pod "c4b48776-3a2c-423c-b695-89de08958a13" (UID: "c4b48776-3a2c-423c-b695-89de08958a13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:17.756689 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.756664    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:17.760990 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.758707    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:17.761319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.761293    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-kube-api-access-wp4xg" pod "c4b48776-3a2c-423c-b695-89de08958a13" (UID: "c4b48776-3a2c-423c-b695-89de08958a13"). InnerVolumeSpecName "kube-api-access-wp4xg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:17.761762 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.761722    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-istio-token" pod "c4b48776-3a2c-423c-b695-89de08958a13" (UID: "c4b48776-3a2c-423c-b695-89de08958a13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:17.769714 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.769657    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"]
Oct 30 18:48:17.819526 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.819498    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-9de872-serving-cert\""
Oct 30 18:48:17.851991 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.851961    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjwwd\" (UniqueName: \"kubernetes.io/projected/bfd0603c-3ee0-476c-916c-e9c4f49b1077-kube-api-access-gjwwd\") pod \"maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:17.852165 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.852007    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-maas-api-tls\") pod \"maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:17.852165 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.852126    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd0603c-3ee0-476c-916c-e9c4f49b1077-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:17.852247 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.852211    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-metrics-certs\") pod \"maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:17.852315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.852303    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4b48776-3a2c-423c-b695-89de08958a13-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:17.852347 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.852324    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:17.852347 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.852334    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:17.852347 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.852345    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:17.852435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.852354    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:17.852435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.852362    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4b48776-3a2c-423c-b695-89de08958a13-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:17.852435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.852370    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:17.852435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.852378    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp4xg\" (UniqueName: \"kubernetes.io/projected/c4b48776-3a2c-423c-b695-89de08958a13-kube-api-access-wp4xg\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:17.852435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.852386    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4b48776-3a2c-423c-b695-89de08958a13-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:17.877017 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.876970    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws" event={"ID":"84ce6a41-5d27-44aa-86bd-b589a00d0c2a","Type":"ContainerStarted","Data":"1b28838d2b76b0d9aa61d845716f30c37ca2f9e94c7dde9c35b56f4b4a767640"}
Oct 30 18:48:17.877225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.877032    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws" event={"ID":"84ce6a41-5d27-44aa-86bd-b589a00d0c2a","Type":"ContainerStarted","Data":"955b99816732d99434447e17cd336a9e768020f939d9528762aeb78a95f6cc54"}
Oct 30 18:48:17.877225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.877172    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 18:48:17.880552 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.880524    4925 generic.go:356] "Generic (PLEG): container finished" podID="c4b48776-3a2c-423c-b695-89de08958a13" containerID="d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329" exitCode=0
Oct 30 18:48:17.880680 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.880609    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"
Oct 30 18:48:17.880740 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.880693    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" event={"ID":"c4b48776-3a2c-423c-b695-89de08958a13","Type":"ContainerDied","Data":"d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329"}
Oct 30 18:48:17.880795 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.880737    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h" event={"ID":"c4b48776-3a2c-423c-b695-89de08958a13","Type":"ContainerDied","Data":"58baf7898eea65ac8ee984eaa6fcf708157a4aea821aa1854be25c80c359ae30"}
Oct 30 18:48:17.880795 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.880769    4925 scope.go:122] "RemoveContainer" containerID="d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329"
Oct 30 18:48:17.892886 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.892872    4925 scope.go:122] "RemoveContainer" containerID="16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228"
Oct 30 18:48:17.894719 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.894682    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws" podStartSLOduration=1.894670254 podStartE2EDuration="1.894670254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:17.893977521 +0000 UTC m=+719.610322599" watchObservedRunningTime="2026-10-30 18:48:17.894670254 +0000 UTC m=+719.611015329"
Oct 30 18:48:17.901392 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.901375    4925 scope.go:122] "RemoveContainer" containerID="d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329"
Oct 30 18:48:17.901659 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:48:17.901641    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329\": container with ID starting with d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329 not found: ID does not exist" containerID="d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329"
Oct 30 18:48:17.901715 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.901667    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329"} err="failed to get container status \"d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329\": rpc error: code = NotFound desc = could not find container \"d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329\": container with ID starting with d3887dab174ce07a1e9e518c3919b8987a8d56d94f2a68319e5301ed69189329 not found: ID does not exist"
Oct 30 18:48:17.901715 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.901682    4925 scope.go:122] "RemoveContainer" containerID="16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228"
Oct 30 18:48:17.901894 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:48:17.901877    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228\": container with ID starting with 16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228 not found: ID does not exist" containerID="16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228"
Oct 30 18:48:17.901930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.901898    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228"} err="failed to get container status \"16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228\": rpc error: code = NotFound desc = could not find container \"16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228\": container with ID starting with 16af8a3282a148a48cb158f13d71fcc22f6535df5a0c7fa958ba5b381672a228 not found: ID does not exist"
Oct 30 18:48:17.913789 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.913765    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"]
Oct 30 18:48:17.920130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.920109    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e4242163-openshift-default-7548d57fc9-fxb8h"]
Oct 30 18:48:17.951194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.951133    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:17.953284 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.953268    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjwwd\" (UniqueName: \"kubernetes.io/projected/bfd0603c-3ee0-476c-916c-e9c4f49b1077-kube-api-access-gjwwd\") pod \"maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:17.953352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.953311    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-maas-api-tls\") pod \"maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:17.953352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.953346    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd0603c-3ee0-476c-916c-e9c4f49b1077-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:17.953462 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.953418    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-metrics-certs\") pod \"maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:17.954165 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.953961    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd0603c-3ee0-476c-916c-e9c4f49b1077-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:17.955917 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.955900    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-maas-api-tls\") pod \"maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:17.956066 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.956046    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-metrics-certs\") pod \"maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:17.961588 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:17.961566    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjwwd\" (UniqueName: \"kubernetes.io/projected/bfd0603c-3ee0-476c-916c-e9c4f49b1077-kube-api-access-gjwwd\") pod \"maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:18.096975 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:18.096941    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:18.166766 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:18.166306    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:18.383818 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:18.383728    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:18.392340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:18.391730    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"]
Oct 30 18:48:18.394032 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:48:18.394007    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd0603c_3ee0_476c_916c_e9c4f49b1077.slice/crio-8d5faf561f0f4faca34a5e529198a12263059a8dfe5fd62e09704ae3982f41cf WatchSource:0}: Error finding container 8d5faf561f0f4faca34a5e529198a12263059a8dfe5fd62e09704ae3982f41cf: Status 404 returned error can't find the container with id 8d5faf561f0f4faca34a5e529198a12263059a8dfe5fd62e09704ae3982f41cf
Oct 30 18:48:18.420600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:18.420579    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56fbb51c-5be2-4b4e-90eb-fe3e81060f80" path="/var/lib/kubelet/pods/56fbb51c-5be2-4b4e-90eb-fe3e81060f80/volumes"
Oct 30 18:48:18.420997 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:18.420981    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f3f5c7e-f689-4f25-9909-7766ebe2ff1e" path="/var/lib/kubelet/pods/9f3f5c7e-f689-4f25-9909-7766ebe2ff1e/volumes"
Oct 30 18:48:18.421394 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:18.421380    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4b48776-3a2c-423c-b695-89de08958a13" path="/var/lib/kubelet/pods/c4b48776-3a2c-423c-b695-89de08958a13/volumes"
Oct 30 18:48:18.574706 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:18.574671    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:18.890516 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:18.890448    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87" event={"ID":"bfd0603c-3ee0-476c-916c-e9c4f49b1077","Type":"ContainerStarted","Data":"c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d"}
Oct 30 18:48:18.890516 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:18.890516    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:18.890938 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:18.890540    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87" event={"ID":"bfd0603c-3ee0-476c-916c-e9c4f49b1077","Type":"ContainerStarted","Data":"8d5faf561f0f4faca34a5e529198a12263059a8dfe5fd62e09704ae3982f41cf"}
Oct 30 18:48:18.910130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:18.910069    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87" podStartSLOduration=1.5156539260000002 podStartE2EDuration="1.910054689s" totalImagesPullingTime="394.400763ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:18.907566513 +0000 UTC m=+720.623911590" watchObservedRunningTime="2026-10-30 18:48:18.910054689 +0000 UTC m=+720.626399765"
Oct 30 18:48:19.160126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.156808    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:19.512929 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.512851    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"]
Oct 30 18:48:19.513314 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.513301    4925 cpu_manager.go:414] "containerMap: removing container" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy"
Oct 30 18:48:19.513364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.513316    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy"
Oct 30 18:48:19.513417 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.513408    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4b48776-3a2c-423c-b695-89de08958a13" containerName="istio-proxy"
Oct 30 18:48:19.516777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.516756    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"
Oct 30 18:48:19.532026 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.531988    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-0458df-dockercfg-xmsrg\""
Oct 30 18:48:19.536779 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.536754    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"]
Oct 30 18:48:19.561139 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.555339    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-0458df\""
Oct 30 18:48:19.564707 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.564669    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:19.573005 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.572961    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwvzc\" (UniqueName: \"kubernetes.io/projected/c83546ba-7aa3-4999-a888-f317b440593b-kube-api-access-dwvzc\") pod \"payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95\" (UID: \"c83546ba-7aa3-4999-a888-f317b440593b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"
Oct 30 18:48:19.573352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.573332    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c83546ba-7aa3-4999-a888-f317b440593b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95\" (UID: \"c83546ba-7aa3-4999-a888-f317b440593b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"
Oct 30 18:48:19.674037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.674003    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c83546ba-7aa3-4999-a888-f317b440593b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95\" (UID: \"c83546ba-7aa3-4999-a888-f317b440593b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"
Oct 30 18:48:19.674210 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.674084    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwvzc\" (UniqueName: \"kubernetes.io/projected/c83546ba-7aa3-4999-a888-f317b440593b-kube-api-access-dwvzc\") pod \"payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95\" (UID: \"c83546ba-7aa3-4999-a888-f317b440593b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"
Oct 30 18:48:19.674681 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.674661    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c83546ba-7aa3-4999-a888-f317b440593b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95\" (UID: \"c83546ba-7aa3-4999-a888-f317b440593b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"
Oct 30 18:48:19.686487 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.686454    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwvzc\" (UniqueName: \"kubernetes.io/projected/c83546ba-7aa3-4999-a888-f317b440593b-kube-api-access-dwvzc\") pod \"payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95\" (UID: \"c83546ba-7aa3-4999-a888-f317b440593b\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"
Oct 30 18:48:19.813486 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.813401    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:19.829361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:19.829333    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"
Oct 30 18:48:20.138736 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.138698    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"]
Oct 30 18:48:20.141769 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:48:20.141739    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc83546ba_7aa3_4999_a888_f317b440593b.slice/crio-d959325347985776bbf24a87451f9a5f213c1f9b8afd0158621081b5adfff50d WatchSource:0}: Error finding container d959325347985776bbf24a87451f9a5f213c1f9b8afd0158621081b5adfff50d: Status 404 returned error can't find the container with id d959325347985776bbf24a87451f9a5f213c1f9b8afd0158621081b5adfff50d
Oct 30 18:48:20.174410 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.174326    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:20.332313 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.332281    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"]
Oct 30 18:48:20.336008 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.335989    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"
Oct 30 18:48:20.345783 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.345688    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-85cb41-dockercfg-5vtfh\""
Oct 30 18:48:20.348636 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.348614    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"]
Oct 30 18:48:20.371286 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.371254    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:20.380519 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.379355    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-85cb41\""
Oct 30 18:48:20.381078 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.381046    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/561dc359-e6c2-4519-b21e-687dffcd3a81-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6\" (UID: \"561dc359-e6c2-4519-b21e-687dffcd3a81\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"
Oct 30 18:48:20.381222 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.381194    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr7qp\" (UniqueName: \"kubernetes.io/projected/561dc359-e6c2-4519-b21e-687dffcd3a81-kube-api-access-xr7qp\") pod \"payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6\" (UID: \"561dc359-e6c2-4519-b21e-687dffcd3a81\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"
Oct 30 18:48:20.486121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.485428    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr7qp\" (UniqueName: \"kubernetes.io/projected/561dc359-e6c2-4519-b21e-687dffcd3a81-kube-api-access-xr7qp\") pod \"payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6\" (UID: \"561dc359-e6c2-4519-b21e-687dffcd3a81\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"
Oct 30 18:48:20.486121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.485611    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/561dc359-e6c2-4519-b21e-687dffcd3a81-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6\" (UID: \"561dc359-e6c2-4519-b21e-687dffcd3a81\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"
Oct 30 18:48:20.488681 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.486497    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/561dc359-e6c2-4519-b21e-687dffcd3a81-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6\" (UID: \"561dc359-e6c2-4519-b21e-687dffcd3a81\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"
Oct 30 18:48:20.496622 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.496590    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr7qp\" (UniqueName: \"kubernetes.io/projected/561dc359-e6c2-4519-b21e-687dffcd3a81-kube-api-access-xr7qp\") pod \"payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6\" (UID: \"561dc359-e6c2-4519-b21e-687dffcd3a81\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"
Oct 30 18:48:20.649911 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.649861    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"
Oct 30 18:48:20.906646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.905837    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95" event={"ID":"c83546ba-7aa3-4999-a888-f317b440593b","Type":"ContainerStarted","Data":"4a3b38c760ff1fb7704209248adec77a3b757f7f6acc5825a313350674f9f5ad"}
Oct 30 18:48:20.906646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.905886    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95" event={"ID":"c83546ba-7aa3-4999-a888-f317b440593b","Type":"ContainerStarted","Data":"d959325347985776bbf24a87451f9a5f213c1f9b8afd0158621081b5adfff50d"}
Oct 30 18:48:20.906646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.906588    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"
Oct 30 18:48:20.925172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.925128    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95" podStartSLOduration=1.925113902 podStartE2EDuration="1.925113902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:20.923157371 +0000 UTC m=+722.639502448" watchObservedRunningTime="2026-10-30 18:48:20.925113902 +0000 UTC m=+722.641458972"
Oct 30 18:48:20.986441 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:20.986407    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"]
Oct 30 18:48:20.987599 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:48:20.987538    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561dc359_e6c2_4519_b21e_687dffcd3a81.slice/crio-ae412e2dce3eb522ae091c32b4eae17bacbcfa310d2e5c715696ef00823c25d9 WatchSource:0}: Error finding container ae412e2dce3eb522ae091c32b4eae17bacbcfa310d2e5c715696ef00823c25d9: Status 404 returned error can't find the container with id ae412e2dce3eb522ae091c32b4eae17bacbcfa310d2e5c715696ef00823c25d9
Oct 30 18:48:21.014192 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.014117    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:21.206786 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.206747    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:21.650034 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.649997    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"]
Oct 30 18:48:21.654719 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.654691    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:21.692122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.679053    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-0458df-serving-cert\""
Oct 30 18:48:21.692122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.690774    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"]
Oct 30 18:48:21.699711 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.698876    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:21.699711 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.698927    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s48pg\" (UniqueName: \"kubernetes.io/projected/dfa60598-c22f-4caa-a3dc-e3092197742f-kube-api-access-s48pg\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:21.699711 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.699002    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfa60598-c22f-4caa-a3dc-e3092197742f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:21.699711 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.699062    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-metrics-certs\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:21.794980 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.794946    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:21.800433 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.800401    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfa60598-c22f-4caa-a3dc-e3092197742f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:21.800582 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.800476    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-metrics-certs\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:21.800582 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.800551    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:21.800715 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.800592    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s48pg\" (UniqueName: \"kubernetes.io/projected/dfa60598-c22f-4caa-a3dc-e3092197742f-kube-api-access-s48pg\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:21.800902 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:48:21.800882    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-serving-cert: secret "maas-api-e2e-worker-w4-0458df-serving-cert" not found
Oct 30 18:48:21.801083 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:48:21.801069    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-maas-api-tls podName:dfa60598-c22f-4caa-a3dc-e3092197742f nodeName:}" failed. No retries permitted until 2026-10-30 18:48:22.301046143 +0000 UTC m=+724.017391203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-maas-api-tls") pod "maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj" (UID: "dfa60598-c22f-4caa-a3dc-e3092197742f") : secret "maas-api-e2e-worker-w4-0458df-serving-cert" not found
Oct 30 18:48:21.801513 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.801486    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfa60598-c22f-4caa-a3dc-e3092197742f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:21.807647 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.805833    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-metrics-certs\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:21.812515 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.812490    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s48pg\" (UniqueName: \"kubernetes.io/projected/dfa60598-c22f-4caa-a3dc-e3092197742f-kube-api-access-s48pg\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:21.913254 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.913219    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6" event={"ID":"561dc359-e6c2-4519-b21e-687dffcd3a81","Type":"ContainerStarted","Data":"88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc"}
Oct 30 18:48:21.913415 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.913364    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"
Oct 30 18:48:21.913477 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.913409    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6" event={"ID":"561dc359-e6c2-4519-b21e-687dffcd3a81","Type":"ContainerStarted","Data":"ae412e2dce3eb522ae091c32b4eae17bacbcfa310d2e5c715696ef00823c25d9"}
Oct 30 18:48:21.934649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.934601    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6" podStartSLOduration=1.934590158 podStartE2EDuration="1.934590158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:21.932946108 +0000 UTC m=+723.649291185" watchObservedRunningTime="2026-10-30 18:48:21.934590158 +0000 UTC m=+723.650935235"
Oct 30 18:48:21.983475 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:21.983447    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:22.271218 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:22.268833    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:22.310414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:22.310373    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:22.313639 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:22.313612    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:22.488767 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:22.488727    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:22.603773 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:22.603675    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:22.688800 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:22.688695    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:22.939313 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:22.938956    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"]
Oct 30 18:48:22.947125 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:48:22.942080    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfa60598_c22f_4caa_a3dc_e3092197742f.slice/crio-05c8ff6f7070a5cedf7376ac91b5d758376df802ab1d40eda00e57e29981f01b WatchSource:0}: Error finding container 05c8ff6f7070a5cedf7376ac91b5d758376df802ab1d40eda00e57e29981f01b: Status 404 returned error can't find the container with id 05c8ff6f7070a5cedf7376ac91b5d758376df802ab1d40eda00e57e29981f01b
Oct 30 18:48:23.440038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:23.439987    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:23.937221 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:23.937184    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj" event={"ID":"dfa60598-c22f-4caa-a3dc-e3092197742f","Type":"ContainerStarted","Data":"48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4"}
Oct 30 18:48:23.938028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:23.938012    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:23.938194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:23.938174    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj" event={"ID":"dfa60598-c22f-4caa-a3dc-e3092197742f","Type":"ContainerStarted","Data":"05c8ff6f7070a5cedf7376ac91b5d758376df802ab1d40eda00e57e29981f01b"}
Oct 30 18:48:23.958925 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:23.957729    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj" podStartSLOduration=2.411085017 podStartE2EDuration="2.957710141s" totalImagesPullingTime="546.625124ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:23.95487917 +0000 UTC m=+725.671224265" watchObservedRunningTime="2026-10-30 18:48:23.957710141 +0000 UTC m=+725.674055220"
Oct 30 18:48:24.269177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:24.266940    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:24.849764 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:24.849737    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"
Oct 30 18:48:24.850145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:24.849808    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"
Oct 30 18:48:24.931865 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:24.931841    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 18:48:25.631676 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:25.631644    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:25.919222 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:25.919185    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:26.677122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:26.676936    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:26.925375 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:26.925329    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:27.250385 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:27.250350    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:27.624696 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:27.624603    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:28.392962 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:28.392858    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:28.894266 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:28.894236    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 18:48:29.365115 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:29.365057    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:30.248914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:30.248875    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:30.575527 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:30.574316    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": read tcp 10.132.0.2:56196->10.132.0.53:15021: read: connection reset by peer" start-of-body=
Oct 30 18:48:30.575527 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:30.574373    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": read tcp 10.132.0.2:56196->10.132.0.53:15021: read: connection reset by peer"
Oct 30 18:48:30.954851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:30.954811    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 18:48:31.023179 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:31.023021    4925 generic.go:356] "Generic (PLEG): container finished" podID="368eed20-8aa8-4387-a513-6be2459125dc" containerID="37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714" exitCode=0
Oct 30 18:48:31.023179 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:31.023128    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" event={"ID":"368eed20-8aa8-4387-a513-6be2459125dc","Type":"ContainerDied","Data":"37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714"}
Oct 30 18:48:31.023601 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:31.023583    4925 scope.go:122] "RemoveContainer" containerID="37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714"
Oct 30 18:48:31.025221 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:31.025182    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:48:31.025352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:31.025277    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:48:31.025352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:31.025326    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:48:31.206876 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:31.206746    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:32.030478 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:32.030438    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" event={"ID":"368eed20-8aa8-4387-a513-6be2459125dc","Type":"ContainerStarted","Data":"d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1"}
Oct 30 18:48:32.091689 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:32.091653    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:32.360406 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:32.360262    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:32.360960 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:32.360934    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 30 18:48:32.361182 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:32.361156    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 30 18:48:32.896809 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:32.896731    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:32.922206 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:32.922179    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"
Oct 30 18:48:32.922372 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:32.922264    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"
Oct 30 18:48:33.365123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:33.360452    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 30 18:48:33.365123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:33.360514    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 30 18:48:33.841221 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:33.841186    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:34.360938 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:34.360437    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 30 18:48:34.360938 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:34.360527    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 30 18:48:34.788428 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:34.788386    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:35.047167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:35.047056    4925 generic.go:356] "Generic (PLEG): container finished" podID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerID="11512d0d37ba8bc25d3d68a508f18025278b9b8f2c345a3462ce9178403bb53d" exitCode=0
Oct 30 18:48:35.047167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:35.047137    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" event={"ID":"49765651-6ef5-47c8-aef6-03b21952a2ad","Type":"ContainerDied","Data":"11512d0d37ba8bc25d3d68a508f18025278b9b8f2c345a3462ce9178403bb53d"}
Oct 30 18:48:35.047589 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:35.047571    4925 scope.go:122] "RemoveContainer" containerID="11512d0d37ba8bc25d3d68a508f18025278b9b8f2c345a3462ce9178403bb53d"
Oct 30 18:48:35.049039 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:35.049008    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:48:35.049178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:35.049082    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:48:35.049178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:35.049139    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:48:35.360605 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:35.360519    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 30 18:48:35.360880 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:35.360852    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 30 18:48:35.728473 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:35.728302    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:36.078180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:36.063067    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" event={"ID":"49765651-6ef5-47c8-aef6-03b21952a2ad","Type":"ContainerStarted","Data":"0ca7a4a6c2d2208f156e85652e4248cb2749b8a70f781d2fd3727804df5e3cce"}
Oct 30 18:48:36.104122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:36.099713    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:36.227120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:36.226498    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:36.227120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:36.226626    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Oct 30 18:48:36.227120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:36.226709    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Oct 30 18:48:36.351382 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:36.351232    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:36.365141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:36.365080    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 30 18:48:36.365323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:36.365167    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 30 18:48:36.553693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:36.553657    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:36.752566 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:36.752395    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:37.226078 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:37.226032    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Oct 30 18:48:37.226498 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:37.226127    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Oct 30 18:48:37.237589 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:37.237561    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:37.417616 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:37.417583    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:37.451912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:37.451881    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:37.453135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:37.453069    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:37.453784 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:37.453751    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 18:48:37.618740 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:37.618654    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:38.043384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:38.043350    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:38.226777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:38.226739    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Oct 30 18:48:38.227186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:38.226795    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Oct 30 18:48:38.298299 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:38.298223    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:38.706757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:38.706707    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:38.911596 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:38.911563    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:39.102425 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:39.102324    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:39.226323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:39.226289    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Oct 30 18:48:39.226479 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:39.226345    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Oct 30 18:48:39.904747 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:39.904704    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:40.226747 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:40.226705    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Oct 30 18:48:40.226927 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:40.226753    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Oct 30 18:48:40.804449 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:40.804416    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:41.226486 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:41.226447    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Oct 30 18:48:41.226970 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:41.226507    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Oct 30 18:48:41.771085 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:41.771040    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:42.231620 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:42.231591    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:42.232250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:42.231995    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:42.232632 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:42.232614    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:48:42.468292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:42.468246    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:43.006694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:43.006666    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:43.236569 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:43.236538    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:43.302029 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:48:43.301946    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:48:43.858896 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:43.858857    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:44.459020 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:44.458988    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:45.303964 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:45.303931    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:45.480736 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:45.480694    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:46.050663 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:46.050637    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:46.322018 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:46.321936    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:46.768314 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:46.768276    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:47.663016 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:47.662982    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:48.451402 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:48.451367    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:49.062836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:49.062797    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:49.743345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:49.743314    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:50.050274 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:50.050171    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:50.550545 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:50.550507    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:51.318948 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:51.318896    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:52.009329 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:52.009293    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:52.639873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:52.639823    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:53.393878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:53.393838    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:54.179432 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:54.179395    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:54.885346 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:54.885300    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:55.586714 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:55.586670    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:56.316119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:56.316052    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:56.617089 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:56.617000    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:57.198435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:57.198392    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:58.013118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:58.013060    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:58.831567 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:58.831476    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:59.650250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:48:59.650212    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:00.829821 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:00.829784    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:01.811592 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:01.811557    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:02.716209 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.716171    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:02.902171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.902136    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"]
Oct 30 18:49:02.906500 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.906467    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:02.929390 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.929362    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9de872\"/\"premium-w0-9de872-kserve-self-signed-certs\""
Oct 30 18:49:02.929582 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.929492    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-9de872\"/\"openshift-service-ca.crt\""
Oct 30 18:49:02.930350 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.930326    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-9de872\"/\"kube-root-ca.crt\""
Oct 30 18:49:02.930600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.930443    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9de872\"/\"default-dockercfg-ddmfm\""
Oct 30 18:49:02.933424 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.933390    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"]
Oct 30 18:49:02.957989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.957955    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-home\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:02.958263 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.958014    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-tmp-dir\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:02.958263 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.958118    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-dshm\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:02.958263 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.958236    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3958fcc7-db32-4565-a937-8fccb3bf644e-tls-certs\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:02.958403 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.958269    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-model-cache\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:02.958403 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:02.958352    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx2c6\" (UniqueName: \"kubernetes.io/projected/3958fcc7-db32-4565-a937-8fccb3bf644e-kube-api-access-xx2c6\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.061131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.061017    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-tmp-dir\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.063118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.063076    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-dshm\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.065932 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.065908    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3958fcc7-db32-4565-a937-8fccb3bf644e-tls-certs\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.066155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.066135    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-model-cache\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.066404 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.066382    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx2c6\" (UniqueName: \"kubernetes.io/projected/3958fcc7-db32-4565-a937-8fccb3bf644e-kube-api-access-xx2c6\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.067055 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.062953    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:03.067055 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.063924    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-tmp-dir\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.067055 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.066801    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-home\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.067055 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.067033    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-model-cache\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.067359 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.066536    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-home\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.071014 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.070989    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3958fcc7-db32-4565-a937-8fccb3bf644e-tls-certs\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.079137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.079040    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-dshm\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.084434 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.084403    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx2c6\" (UniqueName: \"kubernetes.io/projected/3958fcc7-db32-4565-a937-8fccb3bf644e-kube-api-access-xx2c6\") pod \"premium-w0-9de872-kserve-575494f949-5sqdj\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") " pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.234259 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.234226    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:03.479994 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.479958    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:03.523048 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:03.523027    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"]
Oct 30 18:49:03.525126 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:49:03.525079    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3958fcc7_db32_4565_a937_8fccb3bf644e.slice/crio-b7fa7dd1a137df582f66f24330779f33d762573f164ec86aa0ff09c728ce62c8 WatchSource:0}: Error finding container b7fa7dd1a137df582f66f24330779f33d762573f164ec86aa0ff09c728ce62c8: Status 404 returned error can't find the container with id b7fa7dd1a137df582f66f24330779f33d762573f164ec86aa0ff09c728ce62c8
Oct 30 18:49:04.289792 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:04.289746    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" event={"ID":"3958fcc7-db32-4565-a937-8fccb3bf644e","Type":"ContainerStarted","Data":"8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc"}
Oct 30 18:49:04.289792 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:04.289798    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" event={"ID":"3958fcc7-db32-4565-a937-8fccb3bf644e","Type":"ContainerStarted","Data":"b7fa7dd1a137df582f66f24330779f33d762573f164ec86aa0ff09c728ce62c8"}
Oct 30 18:49:04.307652 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:04.307596    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podStartSLOduration=2.3075797270000002 podStartE2EDuration="2.307579727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:04.306678356 +0000 UTC m=+766.023023433" watchObservedRunningTime="2026-10-30 18:49:04.307579727 +0000 UTC m=+766.023924806"
Oct 30 18:49:04.316125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:04.316075    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:05.076707 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:05.076659    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:05.234726 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:05.234689    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:05.771250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:05.771213    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:06.451941 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:06.451903    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:07.018494 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:07.018462    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:08.169929 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:08.169879    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:08.520910 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:08.520804    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:09.150797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:09.150763    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:10.115952 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:10.115918    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:11.176812 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:11.176771    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:12.101687 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:12.101648    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:12.972048 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:12.972015    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:13.234549 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:13.234476    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:13.243054 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:13.243029    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:13.247460 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:13.247440    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:13.306177 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:49:13.306145    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:49:13.350395 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:13.350366    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:49:13.804899 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:13.804867    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:14.865777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:14.865744    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:15.775337 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:15.775294    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:16.779766 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:16.779727    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:17.660426 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:17.660386    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:18.597584 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:18.597550    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:19.167823 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:19.167790    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:19.909385 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:19.909350    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:20.775675 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:20.775637    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:21.488914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:21.488884    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:22.223798 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:22.223765    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:22.972439 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:22.972396    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:23.669220 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:23.669184    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:24.464407 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:24.464368    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:25.555547 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:25.555518    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:26.538810 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:26.538771    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:27.266488 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:27.266449    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:28.152993 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:28.152958    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:29.080879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:29.080841    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:29.679245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:29.679213    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:30.393560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:30.393529    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:31.176509 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:31.176478    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:31.970207 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:31.970171    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:32.677774 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:32.677739    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:33.392153 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:33.392112    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:34.147628 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:34.147594    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:34.926910 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:34.926868    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:35.970975 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:35.970944    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:36.822423 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:36.822384    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:37.594278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.594245    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"]
Oct 30 18:49:37.598551 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.598522    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.602981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.602948    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-2889dd-openshift-default-dockercfg-qgr7b\""
Oct 30 18:49:37.626158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.625246    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"]
Oct 30 18:49:37.696284 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.696246    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:37.770583 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.770552    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz4fp\" (UniqueName: \"kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-kube-api-access-mz4fp\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.770729 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.770591    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.770729 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.770619    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.770729 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.770692    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.770841 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.770751    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.770841 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.770825    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.770904 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.770840    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.770904 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.770888    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.771006 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.770994    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.872327 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.872254    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.872327 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.872308    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz4fp\" (UniqueName: \"kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-kube-api-access-mz4fp\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.872513 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.872343    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.872513 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.872383    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.872513 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.872415    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.872513 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.872441    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.872747 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.872542    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.872747 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.872566    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.872747 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.872592    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.872912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.872816    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.873010 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.872986    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.873203 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.873025    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.873435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.873410    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.873548 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.873431    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.875043 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.875019    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.875155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.875139    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.881032 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.881010    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz4fp\" (UniqueName: \"kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-kube-api-access-mz4fp\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.881300 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.881249    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:37.913202 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:37.913182    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:38.204623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:38.204590    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"]
Oct 30 18:49:38.206757 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:49:38.206731    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb7e1fe_ff9f_456f_a913_8fef2ed627f4.slice/crio-80515137f03a64db9ca354052ef809bc60793515d69101450d1c92c7bda4c298 WatchSource:0}: Error finding container 80515137f03a64db9ca354052ef809bc60793515d69101450d1c92c7bda4c298: Status 404 returned error can't find the container with id 80515137f03a64db9ca354052ef809bc60793515d69101450d1c92c7bda4c298
Oct 30 18:49:38.209628 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:38.209587    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:49:38.209715 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:38.209682    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:49:38.209754 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:38.209729    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:49:38.299880 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:38.299850    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:38.481164 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:38.481046    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" event={"ID":"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4","Type":"ContainerStarted","Data":"8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84"}
Oct 30 18:49:38.481164 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:38.481148    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" event={"ID":"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4","Type":"ContainerStarted","Data":"80515137f03a64db9ca354052ef809bc60793515d69101450d1c92c7bda4c298"}
Oct 30 18:49:38.501737 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:38.501672    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" podStartSLOduration=1.501653667 podStartE2EDuration="1.501653667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:38.499752278 +0000 UTC m=+800.216097380" watchObservedRunningTime="2026-10-30 18:49:38.501653667 +0000 UTC m=+800.217998744"
Oct 30 18:49:38.913786 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:38.913758    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:38.986453 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:38.986418    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:39.768762 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:39.768707    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:39.915317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:39.915281    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 30 18:49:39.915688 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:39.915334    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 30 18:49:40.696022 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:40.695977    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:40.913678 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:40.913635    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 30 18:49:40.913874 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:40.913689    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 30 18:49:41.554419 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:41.554389    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:41.914451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:41.914409    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 30 18:49:41.914647 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:41.914469    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 30 18:49:42.294439 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:42.294144    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:42.914078 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:42.914047    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 30 18:49:42.914489 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:42.914136    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 30 18:49:43.011206 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.011172    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:43.313361 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:49:43.313276    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:49:43.862357 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.862321    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"]
Oct 30 18:49:43.867972 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.867945    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:43.893178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.892811    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-0458df-openshift-default-dockercfg-2vrss\""
Oct 30 18:49:43.896570 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.896504    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:43.898947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.898925    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"]
Oct 30 18:49:43.923146 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.922600    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 30 18:49:43.923146 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.922680    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 30 18:49:43.947805 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.947774    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06139191-a477-4d2b-ab9c-30961ad65838-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:43.947932 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.947841    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:43.947932 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.947907    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:43.947932 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.947929    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06139191-a477-4d2b-ab9c-30961ad65838-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:43.948042 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.948027    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzghv\" (UniqueName: \"kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-kube-api-access-nzghv\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:43.948113 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.948079    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:43.948263 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.948194    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:43.948364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.948281    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:43.948364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:43.948314    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.049750 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.049714    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.049750 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.049750    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.050003 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.049765    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06139191-a477-4d2b-ab9c-30961ad65838-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.050003 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.049956    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzghv\" (UniqueName: \"kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-kube-api-access-nzghv\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.050150 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.050019    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.050150 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.050074    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.050150 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.050122    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.050309 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.050154    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.050309 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.050195    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.050309 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.050210    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06139191-a477-4d2b-ab9c-30961ad65838-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.050518 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.050486    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.050573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.050524    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.050645 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.050621    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.050890 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.050800    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06139191-a477-4d2b-ab9c-30961ad65838-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.052652 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.052632    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06139191-a477-4d2b-ab9c-30961ad65838-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.052735 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.052666    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.064841 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.064818    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzghv\" (UniqueName: \"kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-kube-api-access-nzghv\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.085762 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.085739    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.197720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.197684    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:44.490049 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.490026    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"]
Oct 30 18:49:44.493966 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:49:44.493936    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06139191_a477_4d2b_ab9c_30961ad65838.slice/crio-481c40ac00afd53f96676ad781f3dce24182db271c6a53a75d017f3722af1529 WatchSource:0}: Error finding container 481c40ac00afd53f96676ad781f3dce24182db271c6a53a75d017f3722af1529: Status 404 returned error can't find the container with id 481c40ac00afd53f96676ad781f3dce24182db271c6a53a75d017f3722af1529
Oct 30 18:49:44.496672 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.496628    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:49:44.496775 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.496732    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:49:44.496825 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.496771    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:49:44.542046 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.542012    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch" event={"ID":"06139191-a477-4d2b-ab9c-30961ad65838","Type":"ContainerStarted","Data":"481c40ac00afd53f96676ad781f3dce24182db271c6a53a75d017f3722af1529"}
Oct 30 18:49:44.760741 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.760627    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:44.914828 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.914778    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 30 18:49:44.915008 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:44.914841    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 30 18:49:45.549477 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:45.549441    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch" event={"ID":"06139191-a477-4d2b-ab9c-30961ad65838","Type":"ContainerStarted","Data":"6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af"}
Oct 30 18:49:45.582123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:45.581049    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch" podStartSLOduration=2.581029364 podStartE2EDuration="2.581029364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:45.573620353 +0000 UTC m=+807.289965432" watchObservedRunningTime="2026-10-30 18:49:45.581029364 +0000 UTC m=+807.297374441"
Oct 30 18:49:45.857170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:45.856657    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:46.207120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:46.206861    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:46.209124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:46.207702    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 30 18:49:46.209124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:46.207758    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch" podUID="06139191-a477-4d2b-ab9c-30961ad65838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 30 18:49:46.231783 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:46.231755    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:46.232022 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:46.232001    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:46.232769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:46.232752    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 18:49:46.792125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:46.787036    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:46.972602 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:46.972569    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:47.199050 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:47.199014    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 30 18:49:47.199275 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:47.199077    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch" podUID="06139191-a477-4d2b-ab9c-30961ad65838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 30 18:49:47.747512 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:47.747474    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:48.198789 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:48.198752    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 30 18:49:48.199198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:48.198820    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch" podUID="06139191-a477-4d2b-ab9c-30961ad65838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 30 18:49:48.808674 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:48.808644    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:49.215012 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:49.214976    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 30 18:49:49.219586 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:49.219544    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch" podUID="06139191-a477-4d2b-ab9c-30961ad65838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 30 18:49:50.003124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:50.001656    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:50.198897 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:50.198851    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 30 18:49:50.199131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:50.198930    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch" podUID="06139191-a477-4d2b-ab9c-30961ad65838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 30 18:49:50.782173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:50.782138    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:51.244967 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:51.244924    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:51.245380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:51.245350    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:51.247623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:51.247593    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:49:51.571686 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:51.571611    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:52.275419 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:52.275389    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:52.920852 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:52.920808    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:53.577411 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:53.577371    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:54.288800 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:54.288771    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:55.053933 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:55.053890    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:55.809503 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:55.809462    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:56.647576 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:56.647544    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:57.744728 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:57.744692    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:58.477190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:58.477157    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:59.047533 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.047494    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"]
Oct 30 18:49:59.052037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.052016    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.058422 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.058384    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-85cb41-openshift-default-dockercfg-s655r\""
Oct 30 18:49:59.063777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.063746    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"]
Oct 30 18:49:59.102413 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.102381    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5lwp\" (UniqueName: \"kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-kube-api-access-x5lwp\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.102594 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.102453    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-socket\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.102594 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.102566    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d22fee8-13c0-49d8-8f98-c107c33834e6-istiod-ca-cert\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.102715 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.102636    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-credential-socket\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.102769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.102750    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-envoy\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.102812 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.102784    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-data\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.102861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.102830    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-podinfo\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.102970 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.102954    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-token\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.103032 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.102987    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-certs\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.206311 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.204271    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-data\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.206831 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.206807    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-podinfo\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.207108 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.207072    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-token\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.207262 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.207246    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-certs\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.207445 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.207413    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5lwp\" (UniqueName: \"kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-kube-api-access-x5lwp\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.207609 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.207589    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-socket\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.207688 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.207670    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d22fee8-13c0-49d8-8f98-c107c33834e6-istiod-ca-cert\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.207769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.207753    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-credential-socket\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.207838 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.207820    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-envoy\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.208969 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.207150    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-data\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.209304 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.209278    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-certs\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.210000 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.209970    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d22fee8-13c0-49d8-8f98-c107c33834e6-istiod-ca-cert\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.221133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.217682    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-credential-socket\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.221133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.217761    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-podinfo\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.221133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.217821    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-envoy\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.221133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.217984    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-socket\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.221133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.220290    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-token\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.221133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.220612    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5lwp\" (UniqueName: \"kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-kube-api-access-x5lwp\") pod \"e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") " pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.226837 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.226798    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:59.368314 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.368229    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:49:59.656905 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.656863    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"]
Oct 30 18:49:59.667379 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.667327    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:49:59.667496 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.667475    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:49:59.667597 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.667573    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:49:59.943282 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:49:59.943246    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:00.692235 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:00.692195    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7" event={"ID":"7d22fee8-13c0-49d8-8f98-c107c33834e6","Type":"ContainerStarted","Data":"7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43"}
Oct 30 18:50:00.692750 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:00.692266    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7" event={"ID":"7d22fee8-13c0-49d8-8f98-c107c33834e6","Type":"ContainerStarted","Data":"452370e86070ca9391ad5deddfa7a767df6ea4e77ea5e462fd21d66fff38332a"}
Oct 30 18:50:00.704388 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:00.704361    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:00.716527 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:00.716484    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7" podStartSLOduration=1.716472266 podStartE2EDuration="1.716472266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:00.714802932 +0000 UTC m=+822.431148018" watchObservedRunningTime="2026-10-30 18:50:00.716472266 +0000 UTC m=+822.432817398"
Oct 30 18:50:01.373033 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:01.372999    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:50:01.373381 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:01.373357    4925 patch_prober.go:29] interesting pod/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 30 18:50:01.373524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:01.373500    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7" podUID="7d22fee8-13c0-49d8-8f98-c107c33834e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 30 18:50:01.872625 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:01.872592    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:02.371121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:02.370433    4925 patch_prober.go:29] interesting pod/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 30 18:50:02.371121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:02.370494    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7" podUID="7d22fee8-13c0-49d8-8f98-c107c33834e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 30 18:50:02.719579 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:02.719553    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:02.915406 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:02.915366    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:03.095325 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:03.095232    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:03.371186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:03.369452    4925 patch_prober.go:29] interesting pod/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 30 18:50:03.371186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:03.369522    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7" podUID="7d22fee8-13c0-49d8-8f98-c107c33834e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 30 18:50:03.395863 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:03.395833    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:04.087123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:04.081902    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:04.378177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:04.371452    4925 patch_prober.go:29] interesting pod/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 30 18:50:04.378177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:04.371508    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7" podUID="7d22fee8-13c0-49d8-8f98-c107c33834e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 30 18:50:05.138626 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:05.138577    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:05.374123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:05.372408    4925 patch_prober.go:29] interesting pod/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 30 18:50:05.374123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:05.372470    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7" podUID="7d22fee8-13c0-49d8-8f98-c107c33834e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 30 18:50:06.114370 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:06.114337    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:06.608051 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:06.608015    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:50:06.608508 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:06.608406    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:50:06.609214 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:06.609195    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 18:50:07.144914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:07.144881    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:08.289868 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:08.289836    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:08.551123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:08.551028    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:08.775573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:08.775541    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:09.450722 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:09.450688    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:09.918976 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:09.918945    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:10.396961 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:10.396867    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:11.504807 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:11.504755    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:12.520015 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:12.519982    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:13.303723 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:50:13.303686    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:50:13.442741 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:13.442710    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:14.525481 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:14.525440    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:15.883994 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:15.883954    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:16.612799 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.612769    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"]
Oct 30 18:50:16.624389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.624361    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:16.656889 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.656858    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 18:50:16.678601 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.657251    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-qv9w2\""
Oct 30 18:50:16.678795 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.657313    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 18:50:16.678886 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.657362    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 18:50:16.678961 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.657388    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 18:50:16.679040 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.657652    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-b2c1d1ef4964\""
Oct 30 18:50:16.685069 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.685042    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"]
Oct 30 18:50:16.723189 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.723155    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-routing-overlay-volume\") pod \"payload-processing-external-model-849b7688c9-5fxpt\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") " pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:16.723332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.723265    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txt95\" (UniqueName: \"kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-kube-api-access-txt95\") pod \"payload-processing-external-model-849b7688c9-5fxpt\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") " pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:16.723332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.723290    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-plugins-config-volume\") pod \"payload-processing-external-model-849b7688c9-5fxpt\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") " pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:16.723332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.723309    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-provider-credentials\") pod \"payload-processing-external-model-849b7688c9-5fxpt\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") " pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:16.824878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.824832    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-routing-overlay-volume\") pod \"payload-processing-external-model-849b7688c9-5fxpt\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") " pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:16.825070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.824968    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txt95\" (UniqueName: \"kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-kube-api-access-txt95\") pod \"payload-processing-external-model-849b7688c9-5fxpt\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") " pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:16.825070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.825002    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-plugins-config-volume\") pod \"payload-processing-external-model-849b7688c9-5fxpt\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") " pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:16.825070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.825031    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-provider-credentials\") pod \"payload-processing-external-model-849b7688c9-5fxpt\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") " pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:16.825685 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.825659    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-routing-overlay-volume\") pod \"payload-processing-external-model-849b7688c9-5fxpt\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") " pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:16.825762 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.825715    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-plugins-config-volume\") pod \"payload-processing-external-model-849b7688c9-5fxpt\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") " pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:16.827546 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.827523    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-provider-credentials\") pod \"payload-processing-external-model-849b7688c9-5fxpt\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") " pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:16.839825 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:16.839806    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txt95\" (UniqueName: \"kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-kube-api-access-txt95\") pod \"payload-processing-external-model-849b7688c9-5fxpt\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") " pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:17.009380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:17.009349    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:17.099556 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:17.096954    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:17.344623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:17.344536    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"]
Oct 30 18:50:17.350922 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:50:17.350893    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a7ee665_45c6_4a2d_a538_752ff29ea608.slice/crio-cec20bbd939d48879c645b7f44797a8a87baac01b971723d9b061206cc511f37 WatchSource:0}: Error finding container cec20bbd939d48879c645b7f44797a8a87baac01b971723d9b061206cc511f37: Status 404 returned error can't find the container with id cec20bbd939d48879c645b7f44797a8a87baac01b971723d9b061206cc511f37
Oct 30 18:50:17.834028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:17.825374    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt" event={"ID":"8a7ee665-45c6-4a2d-a538-752ff29ea608","Type":"ContainerStarted","Data":"cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f"}
Oct 30 18:50:17.835520 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:17.834386    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt" event={"ID":"8a7ee665-45c6-4a2d-a538-752ff29ea608","Type":"ContainerStarted","Data":"cec20bbd939d48879c645b7f44797a8a87baac01b971723d9b061206cc511f37"}
Oct 30 18:50:17.835800 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:17.835713    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:17.860255 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:17.860199    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt" podStartSLOduration=1.860184016 podStartE2EDuration="1.860184016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:17.857894613 +0000 UTC m=+839.574239700" watchObservedRunningTime="2026-10-30 18:50:17.860184016 +0000 UTC m=+839.576529092"
Oct 30 18:50:17.884785 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:17.884755    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:18.795144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:18.781313    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:19.079006 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:19.078921    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:19.793681 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:19.793637    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:20.559149 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:20.559081    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:21.387393 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:21.387361    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"]
Oct 30 18:50:21.593443 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:21.593399    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:22.266199 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:22.266162    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:22.971385 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:22.971359    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:23.677146 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:23.676744    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:24.301830 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:24.301785    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:24.971024 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:24.970984    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:25.693208 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:25.693161    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:26.111419 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.111332    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:26.388751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.388637    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt" podUID="8a7ee665-45c6-4a2d-a538-752ff29ea608" containerName="payload-processing" containerID="cri-o://cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f" gracePeriod=25
Oct 30 18:50:26.724885 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.724848    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:26.783129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.782064    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-kube-api-access-txt95\" (UniqueName: \"kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-kube-api-access-txt95\") pod \"8a7ee665-45c6-4a2d-a538-752ff29ea608\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") "
Oct 30 18:50:26.783129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.782197    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-provider-credentials\" (UniqueName: \"kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-provider-credentials\") pod \"8a7ee665-45c6-4a2d-a538-752ff29ea608\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") "
Oct 30 18:50:26.783129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.782252    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-plugins-config-volume\") pod \"8a7ee665-45c6-4a2d-a538-752ff29ea608\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") "
Oct 30 18:50:26.783129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.782278    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-routing-overlay-volume\") pod \"8a7ee665-45c6-4a2d-a538-752ff29ea608\" (UID: \"8a7ee665-45c6-4a2d-a538-752ff29ea608\") "
Oct 30 18:50:26.783457 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.783229    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-plugins-config-volume" pod "8a7ee665-45c6-4a2d-a538-752ff29ea608" (UID: "8a7ee665-45c6-4a2d-a538-752ff29ea608"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:50:26.784487 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.784053    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-routing-overlay-volume" pod "8a7ee665-45c6-4a2d-a538-752ff29ea608" (UID: "8a7ee665-45c6-4a2d-a538-752ff29ea608"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:50:26.786999 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.786979    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-provider-credentials" pod "8a7ee665-45c6-4a2d-a538-752ff29ea608" (UID: "8a7ee665-45c6-4a2d-a538-752ff29ea608"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:50:26.788547 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.788527    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-kube-api-access-txt95" pod "8a7ee665-45c6-4a2d-a538-752ff29ea608" (UID: "8a7ee665-45c6-4a2d-a538-752ff29ea608"). InnerVolumeSpecName "kube-api-access-txt95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:50:26.851116 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.850839    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:26.897955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.897911    4925 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-provider-credentials\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:50:26.898234 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.898210    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:50:26.898367 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.898351    4925 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/8a7ee665-45c6-4a2d-a538-752ff29ea608-routing-overlay-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:50:26.898468 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.898453    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txt95\" (UniqueName: \"kubernetes.io/projected/8a7ee665-45c6-4a2d-a538-752ff29ea608-kube-api-access-txt95\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:50:26.930937 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.930852    4925 generic.go:356] "Generic (PLEG): container finished" podID="8a7ee665-45c6-4a2d-a538-752ff29ea608" containerID="cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f" exitCode=0
Oct 30 18:50:26.931461 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.931229    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"
Oct 30 18:50:26.937861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.931251    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt" event={"ID":"8a7ee665-45c6-4a2d-a538-752ff29ea608","Type":"ContainerDied","Data":"cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f"}
Oct 30 18:50:26.937861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.937426    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt" event={"ID":"8a7ee665-45c6-4a2d-a538-752ff29ea608","Type":"ContainerDied","Data":"cec20bbd939d48879c645b7f44797a8a87baac01b971723d9b061206cc511f37"}
Oct 30 18:50:26.937861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.937467    4925 scope.go:122] "RemoveContainer" containerID="cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f"
Oct 30 18:50:26.952204 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.952186    4925 scope.go:122] "RemoveContainer" containerID="cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f"
Oct 30 18:50:26.952615 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:50:26.952594    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f\": container with ID starting with cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f not found: ID does not exist" containerID="cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f"
Oct 30 18:50:26.952702 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.952626    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f"} err="failed to get container status \"cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f\": rpc error: code = NotFound desc = could not find container \"cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f\": container with ID starting with cb8388e38ca5425b12056b7beaa4f7fc9d0e4fc9b35427d9666f0d3978afd57f not found: ID does not exist"
Oct 30 18:50:26.970136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.969422    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"]
Oct 30 18:50:26.972917 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:26.972893    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-849b7688c9-5fxpt"]
Oct 30 18:50:27.085758 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.085493    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"]
Oct 30 18:50:27.086754 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.086478    4925 cpu_manager.go:414] "containerMap: removing container" podUID="8a7ee665-45c6-4a2d-a538-752ff29ea608" containerName="payload-processing"
Oct 30 18:50:27.086754 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.086555    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a7ee665-45c6-4a2d-a538-752ff29ea608" containerName="payload-processing"
Oct 30 18:50:27.086966 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.086916    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a7ee665-45c6-4a2d-a538-752ff29ea608" containerName="payload-processing"
Oct 30 18:50:27.093173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.093138    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.107838 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.107793    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"]
Oct 30 18:50:27.135540 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.135507    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"]
Oct 30 18:50:27.142178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.140348    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.146009 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.143909    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"]
Oct 30 18:50:27.151223 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.151199    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"]
Oct 30 18:50:27.151347 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.151281    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.220160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.219218    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-workload-certs\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.220160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.219312    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-credential-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.220160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.219359    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90d0440c-184a-473e-897a-8fe6311a2ddf-istio-token\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.220160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.219397    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-istio-data\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.220160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.219439    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-istio-envoy\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.220160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.219469    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90d0440c-184a-473e-897a-8fe6311a2ddf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.220160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.219524    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"]
Oct 30 18:50:27.220160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.219548    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-workload-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.220160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.219577    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90d0440c-184a-473e-897a-8fe6311a2ddf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.220160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.219784    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb6vd\" (UniqueName: \"kubernetes.io/projected/90d0440c-184a-473e-897a-8fe6311a2ddf-kube-api-access-wb6vd\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.320579 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.320544    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.320743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.320605    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-workload-certs\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.320743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.320648    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-credential-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.320743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.320707    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnqwp\" (UniqueName: \"kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-kube-api-access-bnqwp\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.320939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.320745    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-credential-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.320939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.320770    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa78826-8a83-4c64-b87e-642678bf9774-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.320939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.320811    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90d0440c-184a-473e-897a-8fe6311a2ddf-istio-token\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.320939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.320843    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82fwn\" (UniqueName: \"kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-kube-api-access-82fwn\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.320939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.320872    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.320939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.320903    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-istio-data\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.321317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.320939    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-token\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.321317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.320967    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.321317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321002    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-istio-envoy\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.321317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321032    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90d0440c-184a-473e-897a-8fe6311a2ddf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.321317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321065    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.321317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321132    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-certs\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.321317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321167    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.321317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321202    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-envoy\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.321317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321232    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.321317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321267    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.321317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321294    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-data\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.321884 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321325    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa78826-8a83-4c64-b87e-642678bf9774-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.321884 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321366    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-workload-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.321884 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321394    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90d0440c-184a-473e-897a-8fe6311a2ddf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.321884 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321446    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.321884 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321499    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb6vd\" (UniqueName: \"kubernetes.io/projected/90d0440c-184a-473e-897a-8fe6311a2ddf-kube-api-access-wb6vd\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.321884 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321527    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.322247 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.321986    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-workload-certs\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.322327 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.322301    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-credential-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.322737 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.322712    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-workload-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.323489 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.323275    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90d0440c-184a-473e-897a-8fe6311a2ddf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.323489 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.323287    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-istio-data\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.325856 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.325836    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90d0440c-184a-473e-897a-8fe6311a2ddf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.326001 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.325986    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90d0440c-184a-473e-897a-8fe6311a2ddf-istio-envoy\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.330810 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.330790    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90d0440c-184a-473e-897a-8fe6311a2ddf-istio-token\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.330887 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.330853    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb6vd\" (UniqueName: \"kubernetes.io/projected/90d0440c-184a-473e-897a-8fe6311a2ddf-kube-api-access-wb6vd\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-p2rlg\" (UID: \"90d0440c-184a-473e-897a-8fe6311a2ddf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.422477 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422444    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-token\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.422477 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422476    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.422713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422502    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.422713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422531    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-certs\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.422713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422559    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.422713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422586    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-envoy\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.422713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422611    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.422713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422639    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.422713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422660    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-data\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.422713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422684    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa78826-8a83-4c64-b87e-642678bf9774-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.423157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422739    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.423157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422780    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.423157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422815    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.423157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422869    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-credential-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.423157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422922    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnqwp\" (UniqueName: \"kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-kube-api-access-bnqwp\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.423157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.422960    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa78826-8a83-4c64-b87e-642678bf9774-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.423157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.423008    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82fwn\" (UniqueName: \"kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-kube-api-access-82fwn\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.423157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.423039    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.423580 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.423342    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.423580 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.423394    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.423691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.423592    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.423691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.423654    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.423691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.423681    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-certs\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.423853 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.423830    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.424120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.424055    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.424786 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.424424    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-data\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.424786 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.424693    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-credential-socket\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.424936 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.424855    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa78826-8a83-4c64-b87e-642678bf9774-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.426472 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.426438    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-envoy\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.427412 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.427388    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa78826-8a83-4c64-b87e-642678bf9774-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.427605 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.427588    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.427859 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.427838    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.431493 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.431464    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-token\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.432506 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.432478    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.433340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.433310    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnqwp\" (UniqueName: \"kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-kube-api-access-bnqwp\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.433425 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.433343    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82fwn\" (UniqueName: \"kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-kube-api-access-82fwn\") pod \"maas-default-gateway-openshift-default-68cb9b4b95-k9sm4\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.452063 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.452032    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:27.541000 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.540711    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:27.548442 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.548168    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:27.796303 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.784725    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"]
Oct 30 18:50:27.800212 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:50:27.800107    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d0440c_184a_473e_897a_8fe6311a2ddf.slice/crio-fe92d53852e143001c581b1ddc3e16b2075a71611fa2377b08c54791383a2744 WatchSource:0}: Error finding container fe92d53852e143001c581b1ddc3e16b2075a71611fa2377b08c54791383a2744: Status 404 returned error can't find the container with id fe92d53852e143001c581b1ddc3e16b2075a71611fa2377b08c54791383a2744
Oct 30 18:50:27.808882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.802977    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:27.808882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.803056    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:27.808882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.803117    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:27.808882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.804947    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:27.940050 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.940016    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" event={"ID":"90d0440c-184a-473e-897a-8fe6311a2ddf","Type":"ContainerStarted","Data":"3e8bdef2f76134a3f9894bb39b9508725e6564452ba2151d6f7898012bef323a"}
Oct 30 18:50:27.940233 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.940079    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" event={"ID":"90d0440c-184a-473e-897a-8fe6311a2ddf","Type":"ContainerStarted","Data":"fe92d53852e143001c581b1ddc3e16b2075a71611fa2377b08c54791383a2744"}
Oct 30 18:50:27.952796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.952752    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"]
Oct 30 18:50:27.955986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.955717    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"]
Oct 30 18:50:27.968213 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.968162    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" podStartSLOduration=0.968144975 podStartE2EDuration="968.144975ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:27.960429473 +0000 UTC m=+849.676774550" watchObservedRunningTime="2026-10-30 18:50:27.968144975 +0000 UTC m=+849.684490051"
Oct 30 18:50:27.978293 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.977270    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:27.978293 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.977343    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:27.978293 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.977412    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:27.987939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.986273    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:27.987939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.986346    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:27.987939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:27.986388    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:28.426509 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:28.426468    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a7ee665-45c6-4a2d-a538-752ff29ea608" path="/var/lib/kubelet/pods/8a7ee665-45c6-4a2d-a538-752ff29ea608/volumes"
Oct 30 18:50:28.453304 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:28.453275    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:28.454474 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:28.454435    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:50:28.454592 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:28.454506    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" podUID="90d0440c-184a-473e-897a-8fe6311a2ddf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 30 18:50:28.995166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:28.986114    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"]
Oct 30 18:50:29.463127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.423420    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"]
Oct 30 18:50:29.463127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.424243    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.463127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.455384    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:29.463127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.455525    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:50:29.463127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.455579    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" podUID="90d0440c-184a-473e-897a-8fe6311a2ddf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 30 18:50:29.573195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.573150    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x" event={"ID":"1aa78826-8a83-4c64-b87e-642678bf9774","Type":"ContainerStarted","Data":"f6fdc76ecc10a2b164858fd8a6e0509acff9b518212ab8c4a3671a5a287f3f7e"}
Oct 30 18:50:29.573501 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.573474    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x" event={"ID":"1aa78826-8a83-4c64-b87e-642678bf9774","Type":"ContainerStarted","Data":"0607a5d20066171e9490b29a2edc55037cd3adb3b3c0be0abadd732d04fd2a22"}
Oct 30 18:50:29.599864 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.575844    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.600210 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.600179    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5rm7\" (UniqueName: \"kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-kube-api-access-s5rm7\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.600417 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.600400    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01332328-1267-4077-846f-c0553d105634-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.600517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.600503    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01332328-1267-4077-846f-c0553d105634-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.600735 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.600718    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.600848 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.600834    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.600970 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.600957    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.601070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.601056    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.601260 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.601244    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.602583 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.602553    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" event={"ID":"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5","Type":"ContainerStarted","Data":"041d0797ec85b4bc46d103c23097d6da2ef35fd5834ff6b0f32a4a09ac81a751"}
Oct 30 18:50:29.602745 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.602729    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" event={"ID":"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5","Type":"ContainerStarted","Data":"cfd13ec0bc4473595cdf30602fef0c2bc89e325e709924671839b9d320fa9bdf"}
Oct 30 18:50:29.634266 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.603586    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x" podStartSLOduration=2.603568929 podStartE2EDuration="2.603568929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:29.602366982 +0000 UTC m=+851.318712059" watchObservedRunningTime="2026-10-30 18:50:29.603568929 +0000 UTC m=+851.319914006"
Oct 30 18:50:29.649545 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.649478    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" podStartSLOduration=2.649457565 podStartE2EDuration="2.649457565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:29.64862641 +0000 UTC m=+851.364971488" watchObservedRunningTime="2026-10-30 18:50:29.649457565 +0000 UTC m=+851.365802642"
Oct 30 18:50:29.703794 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.703748    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.704044 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.704027    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.704246 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.704226    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.704387 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.704374    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.704555 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.704543    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.704705 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.704690    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.704827 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.704813    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5rm7\" (UniqueName: \"kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-kube-api-access-s5rm7\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.704972 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.704960    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01332328-1267-4077-846f-c0553d105634-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.709118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.705049    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01332328-1267-4077-846f-c0553d105634-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.724355 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.710567    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.724355 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.718488    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.724355 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.718946    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.724355 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.719985    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.729411 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.729382    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.753118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.730491    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01332328-1267-4077-846f-c0553d105634-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.753118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.750666    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01332328-1267-4077-846f-c0553d105634-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.753118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.752805    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.777121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.767821    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5rm7\" (UniqueName: \"kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-kube-api-access-s5rm7\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs\" (UID: \"01332328-1267-4077-846f-c0553d105634\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:29.844362 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:29.844327    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:30.311631 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.311597    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:30.494122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.484468    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:50:30.494122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.484528    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" podUID="90d0440c-184a-473e-897a-8fe6311a2ddf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 30 18:50:30.571427 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.568210    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:30.571427 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.568275    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 18:50:30.571854 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.571743    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:30.571854 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.571793    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x" podUID="1aa78826-8a83-4c64-b87e-642678bf9774" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 18:50:30.643389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.604046    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:30.643389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.604122    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:30.928122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.916399    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"]
Oct 30 18:50:30.952498 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.952462    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:30.952623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.952548    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:30.952623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:30.952592    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:31.468561 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:31.468521    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:50:31.469169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:31.469135    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" podUID="90d0440c-184a-473e-897a-8fe6311a2ddf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 30 18:50:31.551366 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:31.551330    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:31.575122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:31.551612    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 18:50:31.575122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:31.552001    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:31.575122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:31.552039    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x" podUID="1aa78826-8a83-4c64-b87e-642678bf9774" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 18:50:31.767171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:31.759785    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:31.871717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:31.871681    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" event={"ID":"01332328-1267-4077-846f-c0553d105634","Type":"ContainerStarted","Data":"32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c"}
Oct 30 18:50:31.871960 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:31.871935    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" event={"ID":"01332328-1267-4077-846f-c0553d105634","Type":"ContainerStarted","Data":"88f1a182a87ced8c2a4ddc657446e65ef5a904c0916476717fcb99cbaaf29b69"}
Oct 30 18:50:31.914763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:31.914696    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" podStartSLOduration=3.9146751220000002 podStartE2EDuration="3.914675122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:31.906474368 +0000 UTC m=+853.622819424" watchObservedRunningTime="2026-10-30 18:50:31.914675122 +0000 UTC m=+853.631020199"
Oct 30 18:50:32.469166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:32.467671    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:50:32.469166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:32.467728    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" podUID="90d0440c-184a-473e-897a-8fe6311a2ddf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 30 18:50:32.544133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:32.543242    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:32.544133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:32.543305    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 18:50:32.575121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:32.569397    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:32.575121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:32.569454    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x" podUID="1aa78826-8a83-4c64-b87e-642678bf9774" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 18:50:32.847187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:32.846788    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:32.851131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:32.849114    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:50:32.851131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:32.849183    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" podUID="01332328-1267-4077-846f-c0553d105634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 18:50:33.336040 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:33.336001    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:33.452530 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:33.452492    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:50:33.452861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:33.452820    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" podUID="90d0440c-184a-473e-897a-8fe6311a2ddf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 30 18:50:33.562268 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:33.551299    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:33.562268 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:33.551353    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 18:50:33.562268 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:33.552014    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:33.562268 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:33.552053    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x" podUID="1aa78826-8a83-4c64-b87e-642678bf9774" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 18:50:33.868123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:33.867147    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:50:33.868123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:33.867211    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" podUID="01332328-1267-4077-846f-c0553d105634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 18:50:34.457129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:34.456313    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:50:34.457129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:34.456374    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" podUID="90d0440c-184a-473e-897a-8fe6311a2ddf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 30 18:50:34.542118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:34.541486    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:34.542118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:34.541548    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 18:50:34.561119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:34.549424    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:34.561119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:34.549497    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x" podUID="1aa78826-8a83-4c64-b87e-642678bf9774" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 18:50:34.859180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:34.858608    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:50:34.859180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:34.858669    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" podUID="01332328-1267-4077-846f-c0553d105634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 18:50:35.298208 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:35.292813    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:35.453251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:35.453205    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:50:35.453466 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:35.453261    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" podUID="90d0440c-184a-473e-897a-8fe6311a2ddf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 30 18:50:35.542636 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:35.542595    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:35.542836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:35.542661    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 18:50:35.549228 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:35.549050    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 18:50:35.549228 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:35.549124    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x" podUID="1aa78826-8a83-4c64-b87e-642678bf9774" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 18:50:35.846175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:35.845433    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:50:35.846175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:35.845495    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" podUID="01332328-1267-4077-846f-c0553d105634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 18:50:36.465204 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:36.456844    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:50:36.465204 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:36.456919    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" podUID="90d0440c-184a-473e-897a-8fe6311a2ddf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 30 18:50:36.494240 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:36.487770    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:36.552153 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:36.546939    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:36.552153 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:36.547004    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 18:50:36.882174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:36.848462    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:50:36.882174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:36.848564    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" podUID="01332328-1267-4077-846f-c0553d105634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 18:50:36.908121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:36.899048    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:37.361122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:37.357406    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:37.373942 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:37.373907    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:37.424377 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:37.424336    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:50:37.453447 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:37.453380    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 30 18:50:37.453914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:37.453871    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg" podUID="90d0440c-184a-473e-897a-8fe6311a2ddf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 30 18:50:37.541832 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:37.541802    4925 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 18:50:37.542336 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:37.542308    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 18:50:37.845863 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:37.845824    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:50:37.846042 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:37.845874    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" podUID="01332328-1267-4077-846f-c0553d105634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 18:50:37.957186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:37.957148    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:38.846038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:38.846005    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 18:50:38.846667 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:38.846636    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" podUID="01332328-1267-4077-846f-c0553d105634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 18:50:38.974723 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:38.974688    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:39.325141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.325087    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:39.326524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.326498    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:39.329117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.329067    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-p2rlg"
Oct 30 18:50:39.334115 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.334074    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"]
Oct 30 18:50:39.339492 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.339467    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.364141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.362706    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-0458df\"/\"default-dockercfg-ltv6n\""
Oct 30 18:50:39.367545 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.367513    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tmp-dir\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.369637 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.369614    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btk7q\" (UniqueName: \"kubernetes.io/projected/4d4e5708-1d38-4db4-baef-f5863d72c2d1-kube-api-access-btk7q\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.369817 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.367907    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-0458df\"/\"openshift-service-ca.crt\""
Oct 30 18:50:39.370324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.367820    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-0458df\"/\"kube-root-ca.crt\""
Oct 30 18:50:39.370450 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.370019    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tls-certs\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.370610 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.370597    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-home\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.370788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.370775    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-model-cache\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.370931 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.370918    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-dshm\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.375521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.375490    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"]
Oct 30 18:50:39.400602 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.400567    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:39.401804 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.401782    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:39.404017 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.403998    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 18:50:39.467120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.466509    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-0458df\"/\"distinct-2-w4-0458df-kserve-self-signed-certs\""
Oct 30 18:50:39.478778 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.478738    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tls-certs\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.479051 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.479029    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-home\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.479276 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.479258    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-model-cache\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.479402 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.479386    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-dshm\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.479522 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.479509    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tmp-dir\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.479654 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.479639    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btk7q\" (UniqueName: \"kubernetes.io/projected/4d4e5708-1d38-4db4-baef-f5863d72c2d1-kube-api-access-btk7q\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.480985 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.480963    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tmp-dir\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.512646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.481668    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-home\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.512838 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.511831    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-model-cache\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.558520 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.552631    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tls-certs\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.587929 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.556208    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:39.588235 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.585856    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btk7q\" (UniqueName: \"kubernetes.io/projected/4d4e5708-1d38-4db4-baef-f5863d72c2d1-kube-api-access-btk7q\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.595157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.594901    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-dshm\") pod \"distinct-2-w4-0458df-kserve-cddd569df-2szt4\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") " pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:39.690693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:39.690651    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:40.012248 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:40.010511    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"]
Oct 30 18:50:40.015913 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:50:40.015888    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d4e5708_1d38_4db4_baef_f5863d72c2d1.slice/crio-d45c8b0652d63fb08a92dfb978eab37b95df4f69ec135a83fa431e39a0a805c4 WatchSource:0}: Error finding container d45c8b0652d63fb08a92dfb978eab37b95df4f69ec135a83fa431e39a0a805c4: Status 404 returned error can't find the container with id d45c8b0652d63fb08a92dfb978eab37b95df4f69ec135a83fa431e39a0a805c4
Oct 30 18:50:40.081698 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:40.081609    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" event={"ID":"4d4e5708-1d38-4db4-baef-f5863d72c2d1","Type":"ContainerStarted","Data":"d45c8b0652d63fb08a92dfb978eab37b95df4f69ec135a83fa431e39a0a805c4"}
Oct 30 18:50:40.084128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:40.082279    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:40.270058 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:40.269082    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:40.270058 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:40.269531    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:40.270562 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:40.270530    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:50:41.114085 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:41.111155    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" event={"ID":"4d4e5708-1d38-4db4-baef-f5863d72c2d1","Type":"ContainerStarted","Data":"92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286"}
Oct 30 18:50:41.119543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:41.119511    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:41.137125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:41.136564    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podStartSLOduration=2.1365446280000002 podStartE2EDuration="2.136544628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:41.135571875 +0000 UTC m=+862.851916951" watchObservedRunningTime="2026-10-30 18:50:41.136544628 +0000 UTC m=+862.852889706"
Oct 30 18:50:41.692122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:41.691740    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:41.985227 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:41.985137    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:42.877421 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:42.877385    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:43.308217 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:50:43.308152    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:50:43.617334 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:43.617171    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:44.054198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:44.054169    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:44.244574 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:44.244539    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:44.781238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:44.781197    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:45.494485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:45.494450    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:46.027068 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:46.027034    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:46.390202 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:46.387959    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:47.183484 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:47.183450    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:47.864612 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:47.864572    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:48.567186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:48.567147    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:49.281318 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:49.281283    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:49.694600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:49.694569    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:49.807728 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:49.806272    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:50.385864 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:50.385827    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:50.704513 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:50.704479    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:50:52.380376 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:52.380217    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:52.403132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:52.400213    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:50:52.403132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:52.400286    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch" podUID="06139191-a477-4d2b-ab9c-30961ad65838" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:50:52.745871 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:52.745834    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:53.767453 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:53.767406    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:54.077167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.076157    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"]
Oct 30 18:50:54.094477 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.094401    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.098608 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.098583    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"]
Oct 30 18:50:54.113714 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.112756    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-2889dd\"/\"openshift-service-ca.crt\""
Oct 30 18:50:54.113714 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.112830    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-2889dd\"/\"kube-root-ca.crt\""
Oct 30 18:50:54.118710 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.118684    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2889dd\"/\"distinct-2-w3-2889dd-kserve-self-signed-certs\""
Oct 30 18:50:54.118818 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.118722    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2889dd\"/\"default-dockercfg-ptfkg\""
Oct 30 18:50:54.136534 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.136510    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-dshm\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.136677 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.136560    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-home\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.136677 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.136593    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-model-cache\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.136812 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.136738    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ee64038-28e2-4e8e-bb65-2d403a68323c-tls-certs\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.136869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.136806    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-tmp-dir\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.136973 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.136925    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrr84\" (UniqueName: \"kubernetes.io/projected/0ee64038-28e2-4e8e-bb65-2d403a68323c-kube-api-access-zrr84\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.245155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.239274    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ee64038-28e2-4e8e-bb65-2d403a68323c-tls-certs\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.245455 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.245434    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-tmp-dir\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.250553 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.245662    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrr84\" (UniqueName: \"kubernetes.io/projected/0ee64038-28e2-4e8e-bb65-2d403a68323c-kube-api-access-zrr84\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.250829 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.250795    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-dshm\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.253848 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.251044    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-home\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.254933 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.254421    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-home\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.255662 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.246543    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-tmp-dir\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.255916 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.255216    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-model-cache\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.256817 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.255594    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-model-cache\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.262515 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.262490    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-dshm\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.296717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.284036    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ee64038-28e2-4e8e-bb65-2d403a68323c-tls-certs\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.296717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.290368    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrr84\" (UniqueName: \"kubernetes.io/projected/0ee64038-28e2-4e8e-bb65-2d403a68323c-kube-api-access-zrr84\") pod \"distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") " pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.296717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.291273    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:54.419994 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.419952    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:54.711622 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.711597    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"]
Oct 30 18:50:54.713435 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:50:54.713406    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ee64038_28e2_4e8e_bb65_2d403a68323c.slice/crio-01df5fb172ca97c285f2a17a3775b1d9987b20bb3535bc23af65f80fc1d20d00 WatchSource:0}: Error finding container 01df5fb172ca97c285f2a17a3775b1d9987b20bb3535bc23af65f80fc1d20d00: Status 404 returned error can't find the container with id 01df5fb172ca97c285f2a17a3775b1d9987b20bb3535bc23af65f80fc1d20d00
Oct 30 18:50:54.777218 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:54.776001    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:55.471277 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:55.471241    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:55.523489 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:55.523454    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" event={"ID":"0ee64038-28e2-4e8e-bb65-2d403a68323c","Type":"ContainerStarted","Data":"37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82"}
Oct 30 18:50:55.523666 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:55.523497    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" event={"ID":"0ee64038-28e2-4e8e-bb65-2d403a68323c","Type":"ContainerStarted","Data":"01df5fb172ca97c285f2a17a3775b1d9987b20bb3535bc23af65f80fc1d20d00"}
Oct 30 18:50:55.542167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:55.542115    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podStartSLOduration=1.54205907 podStartE2EDuration="1.54205907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:55.540581755 +0000 UTC m=+877.256926839" watchObservedRunningTime="2026-10-30 18:50:55.54205907 +0000 UTC m=+877.258404145"
Oct 30 18:50:56.182598 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:56.182565    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:56.424068 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:56.424027    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:50:56.976655 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:56.974969    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:57.767087 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:57.767018    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:58.466971 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:58.466925    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:59.159987 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:59.159951    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:59.670777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:50:59.670724    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:00.079819 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:00.079677    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:00.867448 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:00.867408    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:01.597649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:01.597584    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:02.272349 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:02.272316    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:02.982124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:02.982011    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:04.071060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:04.071026    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:04.427901 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:04.427873    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:51:04.438908 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:04.438887    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:51:04.640784 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:04.640755    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:51:04.922174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:04.917575    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:05.826360 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:05.826327    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:06.820837 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:06.820792    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:07.555246 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.555212    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:07.714287 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.714251    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"]
Oct 30 18:51:07.718313 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.718287    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.737344 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.737315    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"]
Oct 30 18:51:07.852081 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.852008    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.852081 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.852050    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.852345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.852221    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.852345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.852256    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.852345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.852331    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.852499 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.852351    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.852499 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.852379    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2nqd\" (UniqueName: \"kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-kube-api-access-z2nqd\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.852499 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.852424    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.852639 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.852496    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.953941 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.953909    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.954151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.953948    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.954151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.953976    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2nqd\" (UniqueName: \"kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-kube-api-access-z2nqd\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.954151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.954005    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.954151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.954075    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.954390 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.954225    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.954390 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.954264    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.954390 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.954343    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.954390 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.954380    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.955176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.954470    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.955176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.954502    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.955176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.954592    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.955176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.954749    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.955345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.955188    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.956473 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.956448    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.956731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.956713    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.970941 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.970911    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:07.971057 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:07.971041    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2nqd\" (UniqueName: \"kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-kube-api-access-z2nqd\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:08.049139 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:08.049112    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:08.347601 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:08.347480    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"]
Oct 30 18:51:08.378158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:08.376919    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:08.378158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:08.377009    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:08.378158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:08.377051    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:08.381213 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:08.381138    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:08.583737 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:08.583689    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:08.666610 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:08.666563    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" event={"ID":"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0","Type":"ContainerStarted","Data":"6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483"}
Oct 30 18:51:08.666775 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:08.666640    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" event={"ID":"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0","Type":"ContainerStarted","Data":"3b8f9333290eab4e30cfcf6936a6701df2b7725e42c5c635a13cecebaed2b729"}
Oct 30 18:51:08.690519 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:08.690450    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" podStartSLOduration=1.690432707 podStartE2EDuration="1.690432707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:08.688476597 +0000 UTC m=+890.404821673" watchObservedRunningTime="2026-10-30 18:51:08.690432707 +0000 UTC m=+890.406777785"
Oct 30 18:51:09.049989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:09.049908    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:09.051367 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:09.051341    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:09.051495 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:09.051402    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" podUID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 18:51:09.256121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:09.241769    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:10.051522 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:10.051485    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:10.052215 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:10.052178    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" podUID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 18:51:10.131543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:10.131505    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:10.975123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:10.974607    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:11.050200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:11.050167    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:11.050365 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:11.050224    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" podUID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 18:51:11.788124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:11.787084    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:12.050081 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:12.050013    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:12.050081 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:12.050062    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" podUID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 18:51:12.659237 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:12.659204    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:13.050012 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:13.049925    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:13.050012 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:13.049988    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" podUID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 18:51:13.410160 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:51:13.410120    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:51:13.825545 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:13.825449    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:14.010287 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.010253    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"]
Oct 30 18:51:14.051455 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.051413    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.065983 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.065956    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"]
Oct 30 18:51:14.067125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.066268    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:14.067958 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.067913    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" podUID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 18:51:14.073564 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.073540    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.074356 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.074313    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f1c7961-5121-4119-abaa-5df4bdd09220-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.074679 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.074645    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.074864 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.074836    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.079657 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.075959    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.079864 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.079832    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.080017 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.080000    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.080204 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.080177    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psxpx\" (UniqueName: \"kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-kube-api-access-psxpx\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.080526 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.080505    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.193126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.187736    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.193126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.187789    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.193126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.187830    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.193126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.187874    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psxpx\" (UniqueName: \"kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-kube-api-access-psxpx\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.193126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.187961    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.193126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.188044    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.193126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.191322    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.204119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.196237    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f1c7961-5121-4119-abaa-5df4bdd09220-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.204119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.196284    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.204119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.196330    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.228120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.206001    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.228120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.206916    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f1c7961-5121-4119-abaa-5df4bdd09220-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.228647 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.228619    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.229041 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.228977    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.233970 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.233941    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.269853 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.256433    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.269853 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.269802    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psxpx\" (UniqueName: \"kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-kube-api-access-psxpx\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.308858 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.308805    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.517928 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.517897    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:14.890170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.889049    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:14.913685 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.911918    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"]
Oct 30 18:51:14.927119 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:51:14.925437    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f1c7961_5121_4119_abaa_5df4bdd09220.slice/crio-8337a49e392868786a1dee002d579883042ccb7c1a210417dbf0cd47cfb01f4f WatchSource:0}: Error finding container 8337a49e392868786a1dee002d579883042ccb7c1a210417dbf0cd47cfb01f4f: Status 404 returned error can't find the container with id 8337a49e392868786a1dee002d579883042ccb7c1a210417dbf0cd47cfb01f4f
Oct 30 18:51:14.933272 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.931202    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:14.933272 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.931308    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:14.933272 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:14.931399    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:15.049629 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:15.049598    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 18:51:15.049761 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:15.049659    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" podUID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 18:51:15.072492 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:15.072448    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:15.765122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:15.764382    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6" event={"ID":"2f1c7961-5121-4119-abaa-5df4bdd09220","Type":"ContainerStarted","Data":"4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322"}
Oct 30 18:51:15.765122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:15.764453    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6" event={"ID":"2f1c7961-5121-4119-abaa-5df4bdd09220","Type":"ContainerStarted","Data":"8337a49e392868786a1dee002d579883042ccb7c1a210417dbf0cd47cfb01f4f"}
Oct 30 18:51:15.803120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:15.802935    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6" podStartSLOduration=2.802912089 podStartE2EDuration="2.802912089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:15.791595952 +0000 UTC m=+897.507941032" watchObservedRunningTime="2026-10-30 18:51:15.802912089 +0000 UTC m=+897.519257166"
Oct 30 18:51:15.916135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:15.914566    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:16.444113 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:16.443285    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:16.444113 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:16.444007    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:16.445484 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:16.445455    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:51:16.527121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:16.523464    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:16.540378 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:16.536639    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 18:51:16.540378 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:16.540021    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6" podUID="2f1c7961-5121-4119-abaa-5df4bdd09220" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 18:51:16.921127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:16.919344    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:17.520448 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:17.520411    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 18:51:17.520830 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:17.520476    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6" podUID="2f1c7961-5121-4119-abaa-5df4bdd09220" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 18:51:17.723118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:17.722227    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:18.520285 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:18.520248    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 18:51:18.520491 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:18.520305    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6" podUID="2f1c7961-5121-4119-abaa-5df4bdd09220" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 18:51:18.530370 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:18.530347    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 18:51:18.532655 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:18.532633    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 18:51:18.619395 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:18.619359    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:19.416896 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:19.416847    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:19.520858 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:19.520820    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 18:51:19.521303 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:19.520874    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6" podUID="2f1c7961-5121-4119-abaa-5df4bdd09220" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 18:51:20.303931 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:20.303901    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:20.520043 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:20.520007    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 18:51:20.520210 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:20.520057    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6" podUID="2f1c7961-5121-4119-abaa-5df4bdd09220" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 18:51:21.199194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:21.199155    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:21.519889 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:21.519801    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 18:51:21.519889 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:21.519872    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6" podUID="2f1c7961-5121-4119-abaa-5df4bdd09220" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 18:51:21.779260 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:21.777871    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:22.594130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:22.581254    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:22.599639 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:22.598235    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:22.626897 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:22.626442    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:22.626897 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:22.626744    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 18:51:22.823123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:22.810323    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"]
Oct 30 18:51:22.864591 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:22.863539    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"]
Oct 30 18:51:22.864591 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:22.863678    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.002186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.002154    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.037322 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.037285    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.072264 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.072233    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.078260 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.078233    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mwq2\" (UniqueName: \"kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-kube-api-access-7mwq2\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.131600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.131524    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.131870 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.131850    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.165346 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.165314    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.165617 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.165578    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.165967 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.165947    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68d3f6f9-fa8f-4c43-9921-958e764d442c-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.291117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.290534    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.291117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.290584    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.291117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.290625    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mwq2\" (UniqueName: \"kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-kube-api-access-7mwq2\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.291117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.290658    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.291117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.290703    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.291117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.290743    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.291117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.290789    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.291117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.290821    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68d3f6f9-fa8f-4c43-9921-958e764d442c-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.291117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.290931    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.291674 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.291405    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.291731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.291669    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.292121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.291909    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.309114 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.303928    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.309114 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.304837    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68d3f6f9-fa8f-4c43-9921-958e764d442c-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.374124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.365623    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.374124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.366032    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.374124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.366759    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.378120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.375767    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mwq2\" (UniqueName: \"kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-kube-api-access-7mwq2\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:23.598121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:23.596326    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:24.497870 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:24.497187    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:51:24.983940 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:24.983904    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"]
Oct 30 18:51:25.015633 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:25.015591    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:25.048125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:25.028781    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:25.048491 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:25.048433    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:25.049143 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:25.028669    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:25.209181 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:25.209069    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" event={"ID":"68d3f6f9-fa8f-4c43-9921-958e764d442c","Type":"ContainerStarted","Data":"b47beda01895ad304235a63fe4132f94de4bc1b3880c3992a656f6c935bb657c"}
Oct 30 18:51:25.470175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:25.438718    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:51:26.216545 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.216505    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" event={"ID":"68d3f6f9-fa8f-4c43-9921-958e764d442c","Type":"ContainerStarted","Data":"a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad"}
Oct 30 18:51:26.242152 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.242077    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" podStartSLOduration=4.242059255 podStartE2EDuration="4.242059255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:26.23700071 +0000 UTC m=+907.953345786" watchObservedRunningTime="2026-10-30 18:51:26.242059255 +0000 UTC m=+907.958404331"
Oct 30 18:51:26.597303 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.597195    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:26.598713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.598687    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:51:26.598864 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.598746    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 18:51:26.625291 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.625265    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:26.702183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.694540    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"]
Oct 30 18:51:26.702183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.701219    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:26.722131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.720255    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 18:51:26.722131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.720355    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 18:51:26.722131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.721165    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-1cdb3fee4835\""
Oct 30 18:51:26.756265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.731734    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 18:51:26.756265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.731869    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-qv9w2\""
Oct 30 18:51:26.756265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.731944    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 18:51:26.756265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.740389    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"]
Oct 30 18:51:26.814124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.806004    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:26.814124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.812466    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-provider-credentials\") pod \"payload-processing-external-model-6bc5d5dd85-nvgw6\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") " pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:26.814124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.812544    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-plugins-config-volume\") pod \"payload-processing-external-model-6bc5d5dd85-nvgw6\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") " pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:26.814124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.812598    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx9bw\" (UniqueName: \"kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-kube-api-access-lx9bw\") pod \"payload-processing-external-model-6bc5d5dd85-nvgw6\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") " pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:26.814124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.812640    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-routing-overlay-volume\") pod \"payload-processing-external-model-6bc5d5dd85-nvgw6\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") " pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:26.917132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.914179    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx9bw\" (UniqueName: \"kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-kube-api-access-lx9bw\") pod \"payload-processing-external-model-6bc5d5dd85-nvgw6\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") " pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:26.917132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.914241    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-routing-overlay-volume\") pod \"payload-processing-external-model-6bc5d5dd85-nvgw6\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") " pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:26.917132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.914386    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-provider-credentials\") pod \"payload-processing-external-model-6bc5d5dd85-nvgw6\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") " pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:26.917132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.915654    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-routing-overlay-volume\") pod \"payload-processing-external-model-6bc5d5dd85-nvgw6\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") " pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:26.917132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.915796    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-plugins-config-volume\") pod \"payload-processing-external-model-6bc5d5dd85-nvgw6\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") " pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:26.917132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.917050    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-plugins-config-volume\") pod \"payload-processing-external-model-6bc5d5dd85-nvgw6\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") " pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:26.927804 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.920943    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-provider-credentials\") pod \"payload-processing-external-model-6bc5d5dd85-nvgw6\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") " pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:26.944803 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:26.944765    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx9bw\" (UniqueName: \"kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-kube-api-access-lx9bw\") pod \"payload-processing-external-model-6bc5d5dd85-nvgw6\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") " pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:27.096135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:27.092343    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:27.600552 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:27.600449    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:51:27.600552 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:27.600501    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 18:51:27.639305 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:27.639279    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"]
Oct 30 18:51:27.641296 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:51:27.641268    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b131e5_79aa_4c75_a444_d6e58c29e41b.slice/crio-f3b1c8b066d98499239a660c047e54a5da960331297755859670ce16d908d453 WatchSource:0}: Error finding container f3b1c8b066d98499239a660c047e54a5da960331297755859670ce16d908d453: Status 404 returned error can't find the container with id f3b1c8b066d98499239a660c047e54a5da960331297755859670ce16d908d453
Oct 30 18:51:28.054071 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:28.054036    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:28.347575 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:28.344768    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6" event={"ID":"95b131e5-79aa-4c75-a444-d6e58c29e41b","Type":"ContainerStarted","Data":"549233a898b3c6e9fc51fd39485ccdd04e7bb1a7f4e1cfe41e4f02d0ca51014c"}
Oct 30 18:51:28.347575 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:28.344823    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6" event={"ID":"95b131e5-79aa-4c75-a444-d6e58c29e41b","Type":"ContainerStarted","Data":"f3b1c8b066d98499239a660c047e54a5da960331297755859670ce16d908d453"}
Oct 30 18:51:28.347575 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:28.344868    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:28.597500 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:28.597463    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:51:28.597986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:28.597905    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 18:51:28.843225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:28.841301    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:29.592771 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:29.592725    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:29.597686 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:29.597654    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:51:29.597963 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:29.597931    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 18:51:30.310869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:30.310834    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:30.597651 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:30.597566    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:51:30.597651 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:30.597621    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 18:51:31.167068 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:31.167034    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:31.597809 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:31.597720    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:51:31.597809 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:31.597782    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 18:51:32.062123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:32.058667    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:32.597775 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:32.597731    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 18:51:32.597977 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:32.597801    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 18:51:32.891235 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:32.891050    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:33.794615 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:33.794559    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:33.950408 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:33.950370    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:33.951543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:33.951523    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:33.952545 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:33.952521    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:51:33.979446 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:33.979385    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6" podStartSLOduration=7.97936867 podStartE2EDuration="7.97936867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:28.376375857 +0000 UTC m=+910.092720934" watchObservedRunningTime="2026-10-30 18:51:33.97936867 +0000 UTC m=+915.695713748"
Oct 30 18:51:34.390616 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:34.390577    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:35.056789 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:35.056739    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:35.738503 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:35.738405    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:36.463125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:36.454794    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:37.097304 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:37.095651    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:37.855845 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:37.855810    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:38.514045 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:38.514006    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"]
Oct 30 18:51:38.517493 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:38.517459    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:38.534721 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:38.534691    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:38.723127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:38.716055    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:39.378581 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:39.378541    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:41.242133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:41.241494    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:51:42.322853 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.322811    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:51:42.378928 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.378896    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:51:42.379218 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.379199    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"]
Oct 30 18:51:42.434124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.421840    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:42.460293 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.436363    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 18:51:42.548122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.547509    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:42.548122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.547565    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60687fb8-b28d-474d-9e88-7d919d962b79-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:42.548122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.547621    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9xkx\" (UniqueName: \"kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-kube-api-access-p9xkx\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:42.548122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.547675    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:42.548122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.547735    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:42.548122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.547787    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:42.548122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.547817    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:42.548122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.547870    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:42.548122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:42.548010    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60687fb8-b28d-474d-9e88-7d919d962b79-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.066761 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.066724    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:43.070955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.070928    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"]
Oct 30 18:51:43.071900 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.071882    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"]
Oct 30 18:51:43.120177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.109407    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.124946 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.124916    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.125196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.125172    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60687fb8-b28d-474d-9e88-7d919d962b79-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.125397 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.125378    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9xkx\" (UniqueName: \"kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-kube-api-access-p9xkx\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.125536 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.125520    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.125688 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.125671    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.125836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.125819    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.125968 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.125951    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.126144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.126122    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.126435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.126415    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60687fb8-b28d-474d-9e88-7d919d962b79-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.128343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.128321    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60687fb8-b28d-474d-9e88-7d919d962b79-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.129072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.129048    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.129554 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.129531    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.129959 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.129938    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.190606 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.164272    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.212193 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.199798    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9xkx\" (UniqueName: \"kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-kube-api-access-p9xkx\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.313200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.213320    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:51:43.339124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.213813    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.433763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.427593    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:51:43.433763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.427668    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:51:43.452743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.452669    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.453401 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.453377    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.453633 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.453611    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.453953 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.453933    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9tbk\" (UniqueName: \"kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-kube-api-access-t9tbk\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.454430 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.454389    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.454755 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.454633    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.454994 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.454954    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.455227 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.455208    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.455458 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.455438    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.486574 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.486541    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:43.534121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.512427    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60687fb8-b28d-474d-9e88-7d919d962b79-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.534121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.520289    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6" podUID="95b131e5-79aa-4c75-a444-d6e58c29e41b" containerName="payload-processing" containerID="cri-o://549233a898b3c6e9fc51fd39485ccdd04e7bb1a7f4e1cfe41e4f02d0ca51014c" gracePeriod=25
Oct 30 18:51:43.582694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.558192    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9tbk\" (UniqueName: \"kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-kube-api-access-t9tbk\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.582694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.558246    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.582694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.558299    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.582694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.558351    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.582694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.558399    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.582694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.558443    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.582694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.558495    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.582694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.558583    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.582694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.558612    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.582694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.559266    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.582694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.572010    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.582694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.572309    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.584831 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.583931    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.625255 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.594222    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.625255 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.594285    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"]
Oct 30 18:51:43.647014 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.646474    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.683167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.675798    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.811172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.755673    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:43.847123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.787280    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:43.847123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.814967    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.847123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.829230    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9tbk\" (UniqueName: \"kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-kube-api-access-t9tbk\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.880760 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.880707    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:43.970189 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:43.962787    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:44.204863 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.204827    4925 generic.go:356] "Generic (PLEG): container finished" podID="95b131e5-79aa-4c75-a444-d6e58c29e41b" containerID="549233a898b3c6e9fc51fd39485ccdd04e7bb1a7f4e1cfe41e4f02d0ca51014c" exitCode=0
Oct 30 18:51:44.205331 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.205304    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6" event={"ID":"95b131e5-79aa-4c75-a444-d6e58c29e41b","Type":"ContainerDied","Data":"549233a898b3c6e9fc51fd39485ccdd04e7bb1a7f4e1cfe41e4f02d0ca51014c"}
Oct 30 18:51:44.361120 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:51:44.360314    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:51:44.580120 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:51:44.580033    4925 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b131e5_79aa_4c75_a444_d6e58c29e41b.slice/crio-conmon-549233a898b3c6e9fc51fd39485ccdd04e7bb1a7f4e1cfe41e4f02d0ca51014c.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b131e5_79aa_4c75_a444_d6e58c29e41b.slice/crio-549233a898b3c6e9fc51fd39485ccdd04e7bb1a7f4e1cfe41e4f02d0ca51014c.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 18:51:44.690466 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.688762    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"]
Oct 30 18:51:44.722531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.720636    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:44.722531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.720736    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:44.722531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.720779    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:44.822078 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.821852    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:44.848125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.848080    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-plugins-config-volume\") pod \"95b131e5-79aa-4c75-a444-d6e58c29e41b\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") "
Oct 30 18:51:44.848263 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.848156    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-routing-overlay-volume\") pod \"95b131e5-79aa-4c75-a444-d6e58c29e41b\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") "
Oct 30 18:51:44.848330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.848264    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-kube-api-access-lx9bw\" (UniqueName: \"kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-kube-api-access-lx9bw\") pod \"95b131e5-79aa-4c75-a444-d6e58c29e41b\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") "
Oct 30 18:51:44.848465 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.848405    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-provider-credentials\" (UniqueName: \"kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-provider-credentials\") pod \"95b131e5-79aa-4c75-a444-d6e58c29e41b\" (UID: \"95b131e5-79aa-4c75-a444-d6e58c29e41b\") "
Oct 30 18:51:44.848783 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.848726    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-plugins-config-volume" pod "95b131e5-79aa-4c75-a444-d6e58c29e41b" (UID: "95b131e5-79aa-4c75-a444-d6e58c29e41b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:44.849263 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.849235    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-routing-overlay-volume" pod "95b131e5-79aa-4c75-a444-d6e58c29e41b" (UID: "95b131e5-79aa-4c75-a444-d6e58c29e41b"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:44.851804 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.851721    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-provider-credentials" pod "95b131e5-79aa-4c75-a444-d6e58c29e41b" (UID: "95b131e5-79aa-4c75-a444-d6e58c29e41b"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:51:44.851993 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.851968    4925 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-provider-credentials\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:51:44.852071 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.851996    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:51:44.852071 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.852011    4925 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/95b131e5-79aa-4c75-a444-d6e58c29e41b-routing-overlay-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:51:44.853255 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.853231    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-kube-api-access-lx9bw" pod "95b131e5-79aa-4c75-a444-d6e58c29e41b" (UID: "95b131e5-79aa-4c75-a444-d6e58c29e41b"). InnerVolumeSpecName "kube-api-access-lx9bw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:51:44.942289 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.941921    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:44.981294 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:44.980307    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx9bw\" (UniqueName: \"kubernetes.io/projected/95b131e5-79aa-4c75-a444-d6e58c29e41b-kube-api-access-lx9bw\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:51:45.009492 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.009453    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"]
Oct 30 18:51:45.019615 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:51:45.018886    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32f50b9e_f1e8_47e9_8e50_e3ba166af2de.slice/crio-c6fe51cb36699c163453bdabbcfec430f3f3b3fc4c6969c4cccf709a6b7df3a5 WatchSource:0}: Error finding container c6fe51cb36699c163453bdabbcfec430f3f3b3fc4c6969c4cccf709a6b7df3a5: Status 404 returned error can't find the container with id c6fe51cb36699c163453bdabbcfec430f3f3b3fc4c6969c4cccf709a6b7df3a5
Oct 30 18:51:45.025728 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.023355    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:45.025728 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.023417    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:45.025728 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.023447    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:45.215894 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.215793    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" event={"ID":"60687fb8-b28d-474d-9e88-7d919d962b79","Type":"ContainerStarted","Data":"a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2"}
Oct 30 18:51:45.215894 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.215860    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" event={"ID":"60687fb8-b28d-474d-9e88-7d919d962b79","Type":"ContainerStarted","Data":"af127686693997eae30b7a2c39d73daf33d15659394b981782314e290ccfe78f"}
Oct 30 18:51:45.219164 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.218308    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2" event={"ID":"32f50b9e-f1e8-47e9-8e50-e3ba166af2de","Type":"ContainerStarted","Data":"bf4ec6f99c51191b869e5fa80d3d6a2529efcea12f0da827ba3cf0082c2e8c3f"}
Oct 30 18:51:45.219164 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.218361    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2" event={"ID":"32f50b9e-f1e8-47e9-8e50-e3ba166af2de","Type":"ContainerStarted","Data":"c6fe51cb36699c163453bdabbcfec430f3f3b3fc4c6969c4cccf709a6b7df3a5"}
Oct 30 18:51:45.221792 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.221767    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6" event={"ID":"95b131e5-79aa-4c75-a444-d6e58c29e41b","Type":"ContainerDied","Data":"f3b1c8b066d98499239a660c047e54a5da960331297755859670ce16d908d453"}
Oct 30 18:51:45.221898 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.221825    4925 scope.go:122] "RemoveContainer" containerID="549233a898b3c6e9fc51fd39485ccdd04e7bb1a7f4e1cfe41e4f02d0ca51014c"
Oct 30 18:51:45.221984 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.221970    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"
Oct 30 18:51:45.243853 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.243802    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" podStartSLOduration=3.243784438 podStartE2EDuration="3.243784438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:45.240135053 +0000 UTC m=+926.956480130" watchObservedRunningTime="2026-10-30 18:51:45.243784438 +0000 UTC m=+926.960129515"
Oct 30 18:51:45.255418 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.255395    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"]
Oct 30 18:51:45.262671 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.262643    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-6bc5d5dd85-nvgw6"]
Oct 30 18:51:45.289154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.285762    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2" podStartSLOduration=3.285747144 podStartE2EDuration="3.285747144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:45.281437478 +0000 UTC m=+926.997782596" watchObservedRunningTime="2026-10-30 18:51:45.285747144 +0000 UTC m=+927.002092221"
Oct 30 18:51:45.788346 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.788314    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:45.789202 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.789179    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:45.804763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.802761    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:51:45.804763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.802835    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" podUID="60687fb8-b28d-474d-9e88-7d919d962b79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 18:51:45.973298 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:45.973235    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:46.425086 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:46.425046    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95b131e5-79aa-4c75-a444-d6e58c29e41b" path="/var/lib/kubelet/pods/95b131e5-79aa-4c75-a444-d6e58c29e41b/volumes"
Oct 30 18:51:46.606767 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:46.606727    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:46.792678 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:46.792579    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:51:46.793292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:46.793244    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" podUID="60687fb8-b28d-474d-9e88-7d919d962b79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 18:51:46.973789 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:46.973751    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:51:46.974059 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:46.974026    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2" podUID="32f50b9e-f1e8-47e9-8e50-e3ba166af2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 18:51:47.701120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:47.700926    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:47.795929 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:47.795483    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:51:47.795929 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:47.795546    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" podUID="60687fb8-b28d-474d-9e88-7d919d962b79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 18:51:47.972178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:47.971461    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:51:47.972178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:47.971526    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2" podUID="32f50b9e-f1e8-47e9-8e50-e3ba166af2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 18:51:48.798150 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:48.790484    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:51:48.798150 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:48.790545    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" podUID="60687fb8-b28d-474d-9e88-7d919d962b79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 18:51:48.980121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:48.975668    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:51:48.980121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:48.975727    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2" podUID="32f50b9e-f1e8-47e9-8e50-e3ba166af2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 18:51:49.102205 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:49.097797    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:49.795122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:49.794464    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:51:49.795122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:49.794532    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" podUID="60687fb8-b28d-474d-9e88-7d919d962b79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 18:51:50.037058 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:50.037004    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:51:50.040785 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:50.040749    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2" podUID="32f50b9e-f1e8-47e9-8e50-e3ba166af2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 18:51:50.053811 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:50.053721    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:50.790558 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:50.790516    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:51:50.790917 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:50.790888    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" podUID="60687fb8-b28d-474d-9e88-7d919d962b79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 18:51:50.972482 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:50.972442    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:51:50.972795 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:50.972766    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2" podUID="32f50b9e-f1e8-47e9-8e50-e3ba166af2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 18:51:51.032087 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:51.032050    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:51.792124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:51.788494    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:51:51.792124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:51.788573    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" podUID="60687fb8-b28d-474d-9e88-7d919d962b79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 18:51:51.796117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:51.795329    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:51.971423 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:51.971382    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 18:51:51.971630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:51.971449    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2" podUID="32f50b9e-f1e8-47e9-8e50-e3ba166af2de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 18:51:52.594364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:52.594328    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:52.835921 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:52.835500    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:52.835921 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:52.835866    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:52.837642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:52.837624    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 18:51:52.977046 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:52.977016    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:52.977565 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:52.977504    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:52.978603 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:52.978580    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:51:53.276710 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:53.276620    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:53.880995 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:53.880960    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:54.554423 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:54.554238    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:55.186329 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:55.185797    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:55.987712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:55.987676    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:56.632808 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:56.632776    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"]
Oct 30 18:51:56.634283 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:56.634257    4925 cpu_manager.go:414] "containerMap: removing container" podUID="95b131e5-79aa-4c75-a444-d6e58c29e41b" containerName="payload-processing"
Oct 30 18:51:56.634446 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:56.634430    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95b131e5-79aa-4c75-a444-d6e58c29e41b" containerName="payload-processing"
Oct 30 18:51:56.642137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:56.634648    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="95b131e5-79aa-4c75-a444-d6e58c29e41b" containerName="payload-processing"
Oct 30 18:51:56.794124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:56.774288    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:56.981246 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:56.981206    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.031144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.028802    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"]
Oct 30 18:51:57.164138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.144718    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-9de872\"/\"distinct-2-w0-9de872-kserve-self-signed-certs\""
Oct 30 18:51:57.400145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.399114    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-dshm\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.400145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.399216    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8nkn\" (UniqueName: \"kubernetes.io/projected/82e8999f-b612-48bf-a4ec-c03fb7597b04-kube-api-access-n8nkn\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.400145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.399251    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82e8999f-b612-48bf-a4ec-c03fb7597b04-tls-certs\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.400145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.399325    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-tmp-dir\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.400145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.399385    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-model-cache\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.400145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.399502    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-home\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.532122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.506805    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-dshm\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.533526 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.533494    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8nkn\" (UniqueName: \"kubernetes.io/projected/82e8999f-b612-48bf-a4ec-c03fb7597b04-kube-api-access-n8nkn\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.533717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.533700    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82e8999f-b612-48bf-a4ec-c03fb7597b04-tls-certs\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.533867 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.533854    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-tmp-dir\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.533996 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.533983    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-model-cache\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.534516 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.534493    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-model-cache\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.582521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.534804    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-tmp-dir\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.582521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.534930    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-home\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.582521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.535366    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-home\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.582521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.554337    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-dshm\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.602502 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.602432    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82e8999f-b612-48bf-a4ec-c03fb7597b04-tls-certs\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.613019 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.612768    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8nkn\" (UniqueName: \"kubernetes.io/projected/82e8999f-b612-48bf-a4ec-c03fb7597b04-kube-api-access-n8nkn\") pod \"distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") " pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:57.715758 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:57.715448    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:51:58.960123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:58.952175    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"]
Oct 30 18:51:59.197143 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:51:59.183124    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82e8999f_b612_48bf_a4ec_c03fb7597b04.slice/crio-1d3039a3eeccd0ce8b06bef3989569996a41eb983dddcafc83c7e917dd6afa76 WatchSource:0}: Error finding container 1d3039a3eeccd0ce8b06bef3989569996a41eb983dddcafc83c7e917dd6afa76: Status 404 returned error can't find the container with id 1d3039a3eeccd0ce8b06bef3989569996a41eb983dddcafc83c7e917dd6afa76
Oct 30 18:51:59.247728 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:59.229085    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:51:59.377211 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:51:59.357320    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:00.157177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:00.156548    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" event={"ID":"82e8999f-b612-48bf-a4ec-c03fb7597b04","Type":"ContainerStarted","Data":"1d3039a3eeccd0ce8b06bef3989569996a41eb983dddcafc83c7e917dd6afa76"}
Oct 30 18:52:00.235670 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:00.235615    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:00.241533 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:00.241505    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:52:00.370242 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:00.370199    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:00.371622 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:00.371599    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:52:00.776112 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:00.776062    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:01.088239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:01.088159    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:52:01.099124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:01.099083    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:52:02.396463 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:02.396426    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:02.433961 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:02.433919    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" event={"ID":"82e8999f-b612-48bf-a4ec-c03fb7597b04","Type":"ContainerStarted","Data":"a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6"}
Oct 30 18:52:02.451617 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:02.451558    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podStartSLOduration=6.451544415 podStartE2EDuration="6.451544415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:02.45113056 +0000 UTC m=+944.167475642" watchObservedRunningTime="2026-10-30 18:52:02.451544415 +0000 UTC m=+944.167889491"
Oct 30 18:52:02.716674 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:02.716588    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:52:03.161173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:03.158147    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:03.873135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:03.871576    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:04.569073 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:04.569029    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:05.274144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:05.274090    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:06.021222 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:06.021173    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:06.882993 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:06.882959    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:07.480811 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.480778    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:07.716839 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.716808    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:52:07.729424 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.729400    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:52:07.822259 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.822182    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"]
Oct 30 18:52:07.827421 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.827396    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:07.838993 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.838969    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"]
Oct 30 18:52:07.918288 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.918252    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95570521-ff70-4127-94ba-a717dc1af093-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:07.918751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.918335    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxs7j\" (UniqueName: \"kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-kube-api-access-lxs7j\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:07.918751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.918439    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:07.918751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.918587    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:07.918751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.918625    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:07.918963 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.918750    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:07.918963 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.918807    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:07.918963 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.918897    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95570521-ff70-4127-94ba-a717dc1af093-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:07.918963 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:07.918947    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.019908 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.019878    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95570521-ff70-4127-94ba-a717dc1af093-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.020084 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.019921    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.020084 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.019988    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95570521-ff70-4127-94ba-a717dc1af093-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.020084 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.020014    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxs7j\" (UniqueName: \"kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-kube-api-access-lxs7j\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.020343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.020136    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.020343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.020200    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.020343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.020226    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.020343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.020297    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.020343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.020331    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.020622 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.020597    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.020678 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.020624    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.020756 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.020734    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.020978 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.020771    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.021085 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.021065    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95570521-ff70-4127-94ba-a717dc1af093-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.022751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.022729    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95570521-ff70-4127-94ba-a717dc1af093-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.022834 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.022778    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.028224 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.028198    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxs7j\" (UniqueName: \"kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-kube-api-access-lxs7j\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.028327 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.028258    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.165235 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.157625    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:08.203478 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.203427    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:08.485720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.485689    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"]
Oct 30 18:52:08.490502 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:52:08.490476    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95570521_ff70_4127_94ba_a717dc1af093.slice/crio-dff6567b602a03dcb5027ae91917c26304d75a9fa148608af16e04ac45d459c0 WatchSource:0}: Error finding container dff6567b602a03dcb5027ae91917c26304d75a9fa148608af16e04ac45d459c0: Status 404 returned error can't find the container with id dff6567b602a03dcb5027ae91917c26304d75a9fa148608af16e04ac45d459c0
Oct 30 18:52:08.492925 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.492888    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:52:08.493018 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.492971    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:52:08.493018 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.493008    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:52:08.503034 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.503015    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:52:08.640805 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.639923    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:08.885130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:08.885001    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:09.095668 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:09.095639    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:09.275393 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:09.272984    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:09.498329 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:09.498291    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" event={"ID":"95570521-ff70-4127-94ba-a717dc1af093","Type":"ContainerStarted","Data":"eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c"}
Oct 30 18:52:09.498537 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:09.498356    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" event={"ID":"95570521-ff70-4127-94ba-a717dc1af093","Type":"ContainerStarted","Data":"dff6567b602a03dcb5027ae91917c26304d75a9fa148608af16e04ac45d459c0"}
Oct 30 18:52:09.520274 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:09.520219    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podStartSLOduration=2.520201533 podStartE2EDuration="2.520201533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:09.519172379 +0000 UTC m=+951.235517456" watchObservedRunningTime="2026-10-30 18:52:09.520201533 +0000 UTC m=+951.236546615"
Oct 30 18:52:09.826192 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:09.821874    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:10.191124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:10.180425    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:10.191607 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:10.191185    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:52:10.191607 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:10.191245    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 18:52:10.668695 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:10.668651    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:11.190117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:11.189423    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:52:11.190117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:11.189505    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 18:52:11.521176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:11.519226    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:12.182117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:12.181404    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:52:12.182117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:12.181474    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 18:52:13.196122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:13.192165    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:52:13.196122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:13.192250    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 18:52:13.547175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:13.546482    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:13.613122 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:52:13.612530    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:52:13.773124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:13.772200    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:14.094132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.093985    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:14.109661 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.109610    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:14.143799 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.143762    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"]
Oct 30 18:52:14.231123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.218733    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.231123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.219877    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:52:14.231123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.219939    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 18:52:14.265121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.241356    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:14.425187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.343176    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"]
Oct 30 18:52:14.425187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.383203    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.425187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.383279    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.425187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.383317    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.425187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.383373    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.425187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.383411    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.425187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.383444    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkmrw\" (UniqueName: \"kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-kube-api-access-kkmrw\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.425187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.383481    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12ddc094-da4b-432c-a802-0b21b67e84b8-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.425187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.383542    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.425187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.383677    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.513542 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.513498    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.513858 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.513832    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.517195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.517169    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.521421 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.521386    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.521640 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.521606    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.521790 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.521775    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkmrw\" (UniqueName: \"kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-kube-api-access-kkmrw\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.521939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.521924    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12ddc094-da4b-432c-a802-0b21b67e84b8-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.522143 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.522126    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.522454 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.522437    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.534843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.534807    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.552126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.541862    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12ddc094-da4b-432c-a802-0b21b67e84b8-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.552624 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.552597    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.559178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.552823    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.667080 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.579366    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.667080 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.580057    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkmrw\" (UniqueName: \"kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-kube-api-access-kkmrw\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.667080 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.581920    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:14.667080 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.613629    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:14.669021 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.668986    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:52:14.894181 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.869967    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:14.990590 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:14.990294    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:52:15.040175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:15.039883    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:15.110265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:15.110217    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:15.111080 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:15.111053    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:52:15.232121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:15.211536    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:52:15.232121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:15.211602    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 18:52:15.377121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:15.362048    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:15.445984 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:15.445950    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:15.497340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:15.446208    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:15.497340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:15.496498    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:52:15.497340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:15.461534    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:15.596733 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:15.596662    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:16.019124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:15.998030    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:16.114606 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:16.114562    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:16.226123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:16.204482    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:52:16.226123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:16.204548    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 18:52:16.592655 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:16.511660    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:16.610143 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:16.597266    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:17.030646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:16.998259    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:17.077662 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.077555    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"]
Oct 30 18:52:17.078295 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.078191    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:17.152119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.116240    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:52:17.205724 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.201562    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:52:17.205724 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.201631    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 18:52:17.293175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.291332    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:52:17.293175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.291473    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:52:17.293175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.291536    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:52:17.311390 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.311256    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:52:17.311390 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.311308    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:17.519567 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.519529    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:17.586323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.578659    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:52:17.589987 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.589964    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:52:17.597772 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:17.597720    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:18.047089 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:18.047061    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:52:18.248768 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:18.248672    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:52:18.249010 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:18.248983    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 18:52:18.284579 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:18.284551    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:52:18.521169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:18.510049    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:18.639670 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:18.639620    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" event={"ID":"12ddc094-da4b-432c-a802-0b21b67e84b8","Type":"ContainerStarted","Data":"eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1"}
Oct 30 18:52:18.640342 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:18.639680    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" event={"ID":"12ddc094-da4b-432c-a802-0b21b67e84b8","Type":"ContainerStarted","Data":"2011b413f32480d605f75c6c96a45e28a722c2ad71777d6fd9f62dcf5ec67fc0"}
Oct 30 18:52:19.177844 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:19.177810    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:52:19.178050 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:19.177870    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 18:52:19.382122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:19.381857    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:19.454745 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:19.454652    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:19.455808 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:19.455786    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:52:19.455889 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:19.455834    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 18:52:20.178475 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:20.178434    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 18:52:20.179175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:20.179072    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 18:52:20.293123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:20.292291    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:20.455670 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:20.455586    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:52:20.455935 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:20.455905    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 18:52:21.292266 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:21.292228    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:21.455081 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:21.455041    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:52:21.455291 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:21.455144    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 18:52:22.108121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:22.107297    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:22.228242 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:22.228206    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:52:22.228521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:22.228487    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:22.454919 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:22.454891    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:52:22.455330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:22.454939    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 18:52:22.624564 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:22.623129    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:22.624564 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:22.623735    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:22.624992 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:22.624971    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:52:22.645869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:22.645824    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" podStartSLOduration=8.64581214 podStartE2EDuration="8.64581214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:18.684318524 +0000 UTC m=+960.400663595" watchObservedRunningTime="2026-10-30 18:52:22.64581214 +0000 UTC m=+964.362157216"
Oct 30 18:52:23.034770 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:23.034737    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:23.455317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:23.455289    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:52:23.455705 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:23.455340    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 18:52:23.717734 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:23.717647    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:24.455419 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:24.455388    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:52:24.455974 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:24.455941    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 18:52:25.463406 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:25.462487    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:52:25.463406 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:25.462560    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 18:52:26.393738 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:26.393702    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:26.483143 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:26.476698    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 30 18:52:26.483143 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:26.476863    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 30 18:52:27.149517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.147184    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:52:27.715758 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.715722    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"]
Oct 30 18:52:27.730122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.728820    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"]
Oct 30 18:52:27.730122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.728926    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:27.739904 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.739877    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:27.741671 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.740216    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-85cb41\"/\"default-dockercfg-pfnh7\""
Oct 30 18:52:27.741827 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.740343    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-85cb41\"/\"distinct-2-w1-85cb41-kserve-self-signed-certs\""
Oct 30 18:52:27.741957 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.741943    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:27.747996 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.747978    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:52:27.825690 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.825657    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:27.826432 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.826414    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-85cb41\"/\"openshift-service-ca.crt\""
Oct 30 18:52:27.826769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.826755    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-85cb41\"/\"kube-root-ca.crt\""
Oct 30 18:52:27.960833 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.957749    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-model-cache\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:27.961179 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.961144    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-home\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:27.961365 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.961348    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-dshm\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:27.961553 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.961537    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tmp-dir\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:27.961674 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.961659    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8262m\" (UniqueName: \"kubernetes.io/projected/a51b10af-2839-4268-b3dc-d16dbcd5cd13-kube-api-access-8262m\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:27.961783 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:27.961769    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tls-certs\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.065178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.065047    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tls-certs\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.065425 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.065400    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-model-cache\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.065506 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.065461    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-home\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.065506 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.065483    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-dshm\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.065603 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.065556    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tmp-dir\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.065603 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.065590    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8262m\" (UniqueName: \"kubernetes.io/projected/a51b10af-2839-4268-b3dc-d16dbcd5cd13-kube-api-access-8262m\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.067419 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.067394    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-home\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.068598 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.068563    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-model-cache\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.070574 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.070550    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tmp-dir\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.080165 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.075877    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tls-certs\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.081058 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.080646    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-dshm\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.084338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.084310    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8262m\" (UniqueName: \"kubernetes.io/projected/a51b10af-2839-4268-b3dc-d16dbcd5cd13-kube-api-access-8262m\") pod \"distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") " pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.154117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.154016    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:28.450134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.450086    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"]
Oct 30 18:52:28.451774 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:52:28.451746    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda51b10af_2839_4268_b3dc_d16dbcd5cd13.slice/crio-ea6f0ebfe29ae411576b8a529387bd4e5d621b080f3eb0eac8497815f5f25c58 WatchSource:0}: Error finding container ea6f0ebfe29ae411576b8a529387bd4e5d621b080f3eb0eac8497815f5f25c58: Status 404 returned error can't find the container with id ea6f0ebfe29ae411576b8a529387bd4e5d621b080f3eb0eac8497815f5f25c58
Oct 30 18:52:28.586151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.582293    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:28.586151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.583509    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"]
Oct 30 18:52:28.591650 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.591276    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.606181 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.605644    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"]
Oct 30 18:52:28.704251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.703133    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-credential-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.704251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.703195    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.704251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.703226    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-podinfo\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.704251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.703272    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-envoy\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.704251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.703351    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97223b58-c28a-4e9b-b84c-dbd10acb606e-istiod-ca-cert\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.704251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.703491    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-certs\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.704251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.703574    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-data\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.704251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.703637    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65h8f\" (UniqueName: \"kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-kube-api-access-65h8f\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.704251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.703765    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-token\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.804737 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.804704    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-data\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.805262 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.804784    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65h8f\" (UniqueName: \"kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-kube-api-access-65h8f\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.805262 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.804917    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-token\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.805262 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.805134    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-data\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.805262 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.805195    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-credential-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.805580 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.805264    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.806662 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.805717    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-podinfo\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.806917 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.806890    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-envoy\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.807060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.806991    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97223b58-c28a-4e9b-b84c-dbd10acb606e-istiod-ca-cert\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.807060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.807043    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-certs\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.807452 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.807428    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-certs\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.807552 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.805637    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.807804 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.805589    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-credential-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.808442 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.808418    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97223b58-c28a-4e9b-b84c-dbd10acb606e-istiod-ca-cert\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.809927 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.809765    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-podinfo\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.810824 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.810781    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-envoy\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.814251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.814189    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-token\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.816464 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.816418    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65h8f\" (UniqueName: \"kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-kube-api-access-65h8f\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:28.819455 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.818865    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" event={"ID":"a51b10af-2839-4268-b3dc-d16dbcd5cd13","Type":"ContainerStarted","Data":"75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866"}
Oct 30 18:52:28.819455 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.818923    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" event={"ID":"a51b10af-2839-4268-b3dc-d16dbcd5cd13","Type":"ContainerStarted","Data":"ea6f0ebfe29ae411576b8a529387bd4e5d621b080f3eb0eac8497815f5f25c58"}
Oct 30 18:52:28.838952 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.838909    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podStartSLOduration=1.838897888 podStartE2EDuration="1.838897888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:28.835895353 +0000 UTC m=+970.552240429" watchObservedRunningTime="2026-10-30 18:52:28.838897888 +0000 UTC m=+970.555242963"
Oct 30 18:52:28.960277 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:28.957358    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:29.156905 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:29.155182    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:29.356070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:29.356039    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"]
Oct 30 18:52:29.369317 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:52:29.369258    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97223b58_c28a_4e9b_b84c_dbd10acb606e.slice/crio-eb2e5125fe5db945876ef8737de3504eff4c5b26313b18a9070ec2ad10e05919 WatchSource:0}: Error finding container eb2e5125fe5db945876ef8737de3504eff4c5b26313b18a9070ec2ad10e05919: Status 404 returned error can't find the container with id eb2e5125fe5db945876ef8737de3504eff4c5b26313b18a9070ec2ad10e05919
Oct 30 18:52:29.372547 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:29.372513    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:52:29.372694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:29.372607    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:52:29.372694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:29.372648    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:52:29.541358 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:29.541268    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:29.835317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:29.835265    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" event={"ID":"97223b58-c28a-4e9b-b84c-dbd10acb606e","Type":"ContainerStarted","Data":"82a9fd74b11e9bb36b6752a6556899c390195f40a369b468a1f36fee0bcfc94a"}
Oct 30 18:52:29.835691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:29.835330    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" event={"ID":"97223b58-c28a-4e9b-b84c-dbd10acb606e","Type":"ContainerStarted","Data":"eb2e5125fe5db945876ef8737de3504eff4c5b26313b18a9070ec2ad10e05919"}
Oct 30 18:52:29.857600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:29.857421    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" podStartSLOduration=1.857403865 podStartE2EDuration="1.857403865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:29.8559688 +0000 UTC m=+971.572313875" watchObservedRunningTime="2026-10-30 18:52:29.857403865 +0000 UTC m=+971.573748943"
Oct 30 18:52:29.974394 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:29.974360    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:30.438118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:30.433137    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:30.973121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:30.970228    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 18:52:30.973121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:30.970290    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 18:52:31.203132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:31.195127    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:31.960544 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:31.960509    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 18:52:31.960858 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:31.960826    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 18:52:32.053058 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:32.053023    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:32.905717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:32.905686    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:32.959385 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:32.959349    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 18:52:32.959659 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:32.959627    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 18:52:33.869122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:33.854757    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:33.968223 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:33.968185    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 18:52:33.968530 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:33.968498    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 18:52:34.374631 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:34.374598    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:34.581120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:34.580665    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:34.815118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:34.814632    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:34.970120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:34.960443    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 18:52:34.970120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:34.960511    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 18:52:35.497932 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:35.497896    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:35.714900 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:35.714626    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:35.958239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:35.958203    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 18:52:35.958437 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:35.958271    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 18:52:36.606132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:36.604172    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:36.963043 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:36.963012    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:36.963500 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:36.963473    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:36.964332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:36.964312    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 18:52:37.458791 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:37.458752    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:38.159505 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:38.159473    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:38.199133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:38.198638    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:40.039121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:40.028177    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:40.064121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:40.063221    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:40.565775 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:40.565233    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:40.611123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:40.610192    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:40.694225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:40.694179    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:41.039255 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:41.039202    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:41.065247 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:41.065189    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:41.066221 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:41.066197    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:52:41.118231 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:41.118186    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:52:41.570904 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:41.570849    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:41.595323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:41.595290    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:52:41.658124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:41.633194    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 18:52:41.688124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:41.680564    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:52:42.060123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:42.056677    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 18:52:42.073145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:42.066171    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:42.306833 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:42.305216    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:52:42.326332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:42.308730    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:52:42.505149 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:42.505111    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:42.592187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:42.586448    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:42.699486 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:42.699439    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 18:52:43.213128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:43.198635    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:43.297158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:43.199519    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 18:52:43.309138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:43.306166    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:52:43.595835 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:43.595074    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 18:52:43.787123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:43.784044    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 18:52:43.833871 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:43.833621    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:52:43.833871 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:43.833694    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:43.889184 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:52:43.883366    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:52:44.271122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:44.267209    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:44.285070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:44.283896    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 18:52:44.308612 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:44.308183    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:44.308888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:44.308857    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:52:44.633545 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:44.633445    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:44.886173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:44.873230    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:45.276304 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:45.276249    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:45.302142 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:45.301164    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:45.309244 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:45.309175    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:52:45.481151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:45.474189    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:45.593193 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:45.593074    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:52:45.635226 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:45.635172    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:52:45.885173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:45.881911    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:52:45.923678 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:45.923648    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:52:45.952339 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:45.952310    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:52:45.991357 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:45.991329    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:52:46.523221 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:46.523171    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:47.280736 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:47.280698    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:47.468974 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:47.468937    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:48.107043 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:48.107013    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:48.966884 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:48.965930    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:49.732734 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:49.732702    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:50.377817 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.377779    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:50.561646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.561613    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:50.716480 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.716441    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"]
Oct 30 18:52:50.726466 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.726436    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"]
Oct 30 18:52:50.726641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.726561    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.734956 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.734916    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f37d42-openshift-default-dockercfg-9hc5n\""
Oct 30 18:52:50.816138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.816076    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-data\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.816138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.816124    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-token\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.816364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.816219    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-socket\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.816364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.816260    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-envoy\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.816364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.816293    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-podinfo\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.816519 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.816422    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-certs\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.816519 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.816458    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-credential-socket\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.816623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.816551    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc2gl\" (UniqueName: \"kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-kube-api-access-wc2gl\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.816623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.816582    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e752814-e14b-43f0-9401-71f46afb9a6b-istiod-ca-cert\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.867861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.867826    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:50.917705 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.917674    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-podinfo\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.917839 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.917786    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-certs\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.917839 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.917828    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-credential-socket\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.917920 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.917896    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc2gl\" (UniqueName: \"kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-kube-api-access-wc2gl\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.917955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.917937    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e752814-e14b-43f0-9401-71f46afb9a6b-istiod-ca-cert\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.918039 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.918023    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-data\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.918109 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.918049    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-token\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.918169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.918129    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-socket\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.918169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.918153    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-envoy\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.918424 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.918397    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-certs\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.918592 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.918568    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-data\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.918692 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.918579    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-credential-socket\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.918692 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.918605    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-socket\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.918913 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.918895    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e752814-e14b-43f0-9401-71f46afb9a6b-istiod-ca-cert\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.920713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.920695    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-podinfo\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.921401 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.921380    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-envoy\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.926451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.926425    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-token\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:50.926626 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:50.926602    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc2gl\" (UniqueName: \"kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-kube-api-access-wc2gl\") pod \"e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") " pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:51.063174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:51.061421    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:51.068753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:51.068646    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:51.493694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:51.493319    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:51.556822 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:51.556791    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"]
Oct 30 18:52:51.558391 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:52:51.558358    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e752814_e14b_43f0_9401_71f46afb9a6b.slice/crio-f4ddad96139e97835f304b121b1a3b340c06b18156a79b60eedd186de16c2355 WatchSource:0}: Error finding container f4ddad96139e97835f304b121b1a3b340c06b18156a79b60eedd186de16c2355: Status 404 returned error can't find the container with id f4ddad96139e97835f304b121b1a3b340c06b18156a79b60eedd186de16c2355
Oct 30 18:52:51.560819 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:51.560784    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:52:51.560933 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:51.560861    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:52:51.560997 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:51.560939    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:52:51.667415 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:51.667346    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:52.062161 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:52.061740    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:52.242131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:52.242047    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:52.329207 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:52.329059    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" event={"ID":"4e752814-e14b-43f0-9401-71f46afb9a6b","Type":"ContainerStarted","Data":"4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c"}
Oct 30 18:52:52.329207 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:52.329164    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" event={"ID":"4e752814-e14b-43f0-9401-71f46afb9a6b","Type":"ContainerStarted","Data":"f4ddad96139e97835f304b121b1a3b340c06b18156a79b60eedd186de16c2355"}
Oct 30 18:52:52.358049 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:52.357986    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" podStartSLOduration=2.357972538 podStartE2EDuration="2.357972538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:52.357401057 +0000 UTC m=+994.073746134" watchObservedRunningTime="2026-10-30 18:52:52.357972538 +0000 UTC m=+994.074317610"
Oct 30 18:52:52.569514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:52.569476    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:53.104010 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:53.092494    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:53.250722 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:53.249282    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:53.336126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:53.336079    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:53.337027 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:53.337006    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:52:53.557153 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:53.554633    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:54.027819 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:54.027786    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:54.656047 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:54.656015    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:54.957458 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:54.957414    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:55.155184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:55.155152    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:55.373887 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:55.373780    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:55.566549 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:55.565649    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:55.772809 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:55.772771    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:56.097791 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.097711    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"]
Oct 30 18:52:56.102451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.102428    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"
Oct 30 18:52:56.108515 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.108488    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-f37d42\""
Oct 30 18:52:56.110065 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.109725    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-f37d42-dockercfg-g5xcg\""
Oct 30 18:52:56.114793 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.114769    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"]
Oct 30 18:52:56.128988 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.128964    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"]
Oct 30 18:52:56.133227 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.133205    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"
Oct 30 18:52:56.142719 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.141615    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"]
Oct 30 18:52:56.156186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.156154    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:56.175173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.175146    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6r2l\" (UniqueName: \"kubernetes.io/projected/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-kube-api-access-s6r2l\") pod \"payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4\" (UID: \"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"
Oct 30 18:52:56.175581 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.175554    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlmh4\" (UniqueName: \"kubernetes.io/projected/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-kube-api-access-dlmh4\") pod \"payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d\" (UID: \"9a7ef085-d27e-4a4c-8ca7-ff28e7175777\") " pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"
Oct 30 18:52:56.175805 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.175785    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4\" (UID: \"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"
Oct 30 18:52:56.175975 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.175957    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d\" (UID: \"9a7ef085-d27e-4a4c-8ca7-ff28e7175777\") " pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"
Oct 30 18:52:56.295129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.284175    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4\" (UID: \"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"
Oct 30 18:52:56.295129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.284260    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d\" (UID: \"9a7ef085-d27e-4a4c-8ca7-ff28e7175777\") " pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"
Oct 30 18:52:56.295129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.284337    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6r2l\" (UniqueName: \"kubernetes.io/projected/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-kube-api-access-s6r2l\") pod \"payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4\" (UID: \"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"
Oct 30 18:52:56.295129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.284517    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlmh4\" (UniqueName: \"kubernetes.io/projected/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-kube-api-access-dlmh4\") pod \"payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d\" (UID: \"9a7ef085-d27e-4a4c-8ca7-ff28e7175777\") " pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"
Oct 30 18:52:56.308121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.299252    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d\" (UID: \"9a7ef085-d27e-4a4c-8ca7-ff28e7175777\") " pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"
Oct 30 18:52:56.323525 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.310650    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4\" (UID: \"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"
Oct 30 18:52:56.361240 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.333794    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlmh4\" (UniqueName: \"kubernetes.io/projected/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-kube-api-access-dlmh4\") pod \"payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d\" (UID: \"9a7ef085-d27e-4a4c-8ca7-ff28e7175777\") " pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"
Oct 30 18:52:56.361240 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.341544    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:56.365059 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.365030    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6r2l\" (UniqueName: \"kubernetes.io/projected/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-kube-api-access-s6r2l\") pod \"payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4\" (UID: \"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"
Oct 30 18:52:56.454784 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.454508    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"
Oct 30 18:52:56.465223 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.458641    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"
Oct 30 18:52:56.792612 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.792578    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"]
Oct 30 18:52:56.802671 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:52:56.802641    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaf7a6bc_5f1e_42b3_ab9c_da01dde9cdd5.slice/crio-34da69742c7088cebfa2437ae7e34bc7611d33257ab2f317ff670ffbcca4d14d WatchSource:0}: Error finding container 34da69742c7088cebfa2437ae7e34bc7611d33257ab2f317ff670ffbcca4d14d: Status 404 returned error can't find the container with id 34da69742c7088cebfa2437ae7e34bc7611d33257ab2f317ff670ffbcca4d14d
Oct 30 18:52:56.814223 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.814199    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"]
Oct 30 18:52:56.816229 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:52:56.816208    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a7ef085_d27e_4a4c_8ca7_ff28e7175777.slice/crio-96c1b05d9a85679ab0c3b6694429123693e43241c637e712136cad97c182fe1b WatchSource:0}: Error finding container 96c1b05d9a85679ab0c3b6694429123693e43241c637e712136cad97c182fe1b: Status 404 returned error can't find the container with id 96c1b05d9a85679ab0c3b6694429123693e43241c637e712136cad97c182fe1b
Oct 30 18:52:56.929139 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:56.925408    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:57.092907 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.092820    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"]
Oct 30 18:52:57.098056 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.098024    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"
Oct 30 18:52:57.107617 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.107473    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-87293f-dockercfg-zgwpp\""
Oct 30 18:52:57.107617 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.107541    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-87293f\""
Oct 30 18:52:57.109458 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.109072    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"]
Oct 30 18:52:57.160198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.160164    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:57.287121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.286301    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn\" (UID: \"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"
Oct 30 18:52:57.287121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.286405    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glm8d\" (UniqueName: \"kubernetes.io/projected/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-kube-api-access-glm8d\") pod \"payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn\" (UID: \"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"
Oct 30 18:52:57.361743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.361654    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:57.388202 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.388161    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn\" (UID: \"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"
Oct 30 18:52:57.388457 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.388438    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glm8d\" (UniqueName: \"kubernetes.io/projected/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-kube-api-access-glm8d\") pod \"payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn\" (UID: \"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"
Oct 30 18:52:57.390027 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.390003    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn\" (UID: \"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"
Oct 30 18:52:57.425121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.408333    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glm8d\" (UniqueName: \"kubernetes.io/projected/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-kube-api-access-glm8d\") pod \"payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn\" (UID: \"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb\") " pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"
Oct 30 18:52:57.425121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.418384    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"
Oct 30 18:52:57.520036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.514671    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4" event={"ID":"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5","Type":"ContainerStarted","Data":"b5b76adafc084a4c561fcbeeaf6a8154cf1ffbab4e81ede41337dc55be106ddc"}
Oct 30 18:52:57.520036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.514752    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4" event={"ID":"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5","Type":"ContainerStarted","Data":"34da69742c7088cebfa2437ae7e34bc7611d33257ab2f317ff670ffbcca4d14d"}
Oct 30 18:52:57.520036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.515539    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"
Oct 30 18:52:57.520036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.518373    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d" event={"ID":"9a7ef085-d27e-4a4c-8ca7-ff28e7175777","Type":"ContainerStarted","Data":"407125055fe39c3f587c44bee6db6b1aee97880352961469c2e858c3600ec05a"}
Oct 30 18:52:57.520036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.518412    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d" event={"ID":"9a7ef085-d27e-4a4c-8ca7-ff28e7175777","Type":"ContainerStarted","Data":"96c1b05d9a85679ab0c3b6694429123693e43241c637e712136cad97c182fe1b"}
Oct 30 18:52:57.520036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.518915    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"
Oct 30 18:52:57.539990 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.539933    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4" podStartSLOduration=1.539916067 podStartE2EDuration="1.539916067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:57.532262876 +0000 UTC m=+999.248607954" watchObservedRunningTime="2026-10-30 18:52:57.539916067 +0000 UTC m=+999.256261143"
Oct 30 18:52:57.565511 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.565457    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d" podStartSLOduration=1.565444724 podStartE2EDuration="1.565444724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:57.553190284 +0000 UTC m=+999.269535360" watchObservedRunningTime="2026-10-30 18:52:57.565444724 +0000 UTC m=+999.281789802"
Oct 30 18:52:57.788907 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.785362    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"]
Oct 30 18:52:57.942117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:57.939786    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:58.530480 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:58.530444    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn" event={"ID":"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb","Type":"ContainerStarted","Data":"002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904"}
Oct 30 18:52:58.530662 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:58.530516    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"
Oct 30 18:52:58.530662 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:58.530538    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn" event={"ID":"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb","Type":"ContainerStarted","Data":"36870be960dd7a0994b8cbb1ff0f46eb6dd878665d328f4d0150cdb78fadec04"}
Oct 30 18:52:58.550953 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:58.550907    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn" podStartSLOduration=1.550892051 podStartE2EDuration="1.550892051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:58.548485497 +0000 UTC m=+1000.264830574" watchObservedRunningTime="2026-10-30 18:52:58.550892051 +0000 UTC m=+1000.267237127"
Oct 30 18:52:58.610505 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:58.610471    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:59.236401 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:59.236368    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:59.854028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:59.853990    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"]
Oct 30 18:52:59.859697 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:59.859668    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:52:59.864643 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:59.864597    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f37d42-serving-cert\""
Oct 30 18:52:59.866968 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:59.866942    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"]
Oct 30 18:52:59.962224 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:59.962191    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-metrics-certs\") pod \"maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:52:59.962395 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:59.962372    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25d54e45-58d2-4476-869c-4b076b617038-openshift-service-ca\") pod \"maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:52:59.962509 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:59.962489    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7cdv\" (UniqueName: \"kubernetes.io/projected/25d54e45-58d2-4476-869c-4b076b617038-kube-api-access-p7cdv\") pod \"maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:52:59.962623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:59.962608    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-maas-api-tls\") pod \"maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:52:59.983384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:52:59.983357    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:00.065080 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:00.065040    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-metrics-certs\") pod \"maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:53:00.082648 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:00.082612    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25d54e45-58d2-4476-869c-4b076b617038-openshift-service-ca\") pod \"maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:53:00.082946 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:00.082927    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7cdv\" (UniqueName: \"kubernetes.io/projected/25d54e45-58d2-4476-869c-4b076b617038-kube-api-access-p7cdv\") pod \"maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:53:00.083150 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:00.083133    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-maas-api-tls\") pod \"maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:53:00.091239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:00.087769    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-maas-api-tls\") pod \"maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:53:00.091239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:00.077914    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-metrics-certs\") pod \"maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:53:00.091239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:00.088572    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25d54e45-58d2-4476-869c-4b076b617038-openshift-service-ca\") pod \"maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:53:00.103306 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:00.103259    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7cdv\" (UniqueName: \"kubernetes.io/projected/25d54e45-58d2-4476-869c-4b076b617038-kube-api-access-p7cdv\") pod \"maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:53:00.187119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:00.180861    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:00.208834 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:00.208759    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:53:00.368122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:00.358782    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:01.375123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:01.337177    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:01.377407 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:01.376730    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:01.602123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:01.574138    4925 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:53:01.620425 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:01.599376    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"]
Oct 30 18:53:01.725060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:01.725010    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 18:53:01.727335 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:01.725637    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f" event={"ID":"25d54e45-58d2-4476-869c-4b076b617038","Type":"ContainerStarted","Data":"a6cd493f2fa828ba547cb6ef82bc52ecb6b3539bbca706fd422555c58d938ab3"}
Oct 30 18:53:01.833289 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:01.833256    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:02.149180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.148792    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:02.185118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.184243    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 18:53:02.233124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.228143    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"]
Oct 30 18:53:02.283122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.270208    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:02.343193 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.322835    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-87293f-serving-cert\""
Oct 30 18:53:02.368124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.349359    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:53:02.368124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.350086    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:53:02.368124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.361442    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffa5c77a-9830-4201-8b01-47edec592238-openshift-service-ca\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:02.368124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.361513    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:02.368124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.361566    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b22xf\" (UniqueName: \"kubernetes.io/projected/ffa5c77a-9830-4201-8b01-47edec592238-kube-api-access-b22xf\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:02.368124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.361666    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-metrics-certs\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:02.404172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.388927    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 18:53:02.404172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.391221    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:53:02.404172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.403563    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:02.404172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.403673    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:53:02.457123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.428183    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"]
Oct 30 18:53:02.514661 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.514625    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:02.544134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.535030    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffa5c77a-9830-4201-8b01-47edec592238-openshift-service-ca\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:02.544134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.535125    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:02.544134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.535175    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b22xf\" (UniqueName: \"kubernetes.io/projected/ffa5c77a-9830-4201-8b01-47edec592238-kube-api-access-b22xf\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:02.544134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.535313    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-metrics-certs\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:02.579345 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:53:02.579274    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-serving-cert: secret "maas-api-e2e-api-a-87293f-serving-cert" not found
Oct 30 18:53:02.579560 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:53:02.579362    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls podName:ffa5c77a-9830-4201-8b01-47edec592238 nodeName:}" failed. No retries permitted until 2026-10-30 18:53:03.079338737 +0000 UTC m=+1004.795683808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls") pod "maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85" (UID: "ffa5c77a-9830-4201-8b01-47edec592238") : secret "maas-api-e2e-api-a-87293f-serving-cert" not found
Oct 30 18:53:02.692060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.649175    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 18:53:02.710135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.706365    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffa5c77a-9830-4201-8b01-47edec592238-openshift-service-ca\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:02.837232 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.753183    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 18:53:02.837232 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.753877    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:53:02.898078 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.885849    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:02.994632 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:02.973130    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b22xf\" (UniqueName: \"kubernetes.io/projected/ffa5c77a-9830-4201-8b01-47edec592238-kube-api-access-b22xf\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:03.131133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:03.127898    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-metrics-certs\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:03.156436 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:03.129268    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:03.156843 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:53:03.129375    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-serving-cert: secret "maas-api-e2e-api-a-87293f-serving-cert" not found
Oct 30 18:53:03.157071 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:53:03.157057    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls podName:ffa5c77a-9830-4201-8b01-47edec592238 nodeName:}" failed. No retries permitted until 2026-10-30 18:53:04.157018064 +0000 UTC m=+1005.873363158 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls") pod "maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85" (UID: "ffa5c77a-9830-4201-8b01-47edec592238") : secret "maas-api-e2e-api-a-87293f-serving-cert" not found
Oct 30 18:53:03.187292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:03.186111    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:03.192129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:03.189203    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:53:03.375179 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:03.373271    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:53:03.512125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:03.471027    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 18:53:03.695548 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:03.674584    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:03.744122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:03.713737    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:53:03.744122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:03.713803    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:53:04.243161 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:04.237554    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 18:53:04.290391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:04.290354    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:04.329121 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:53:04.302522    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-serving-cert: secret "maas-api-e2e-api-a-87293f-serving-cert" not found
Oct 30 18:53:04.329121 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:53:04.302611    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls podName:ffa5c77a-9830-4201-8b01-47edec592238 nodeName:}" failed. No retries permitted until 2026-10-30 18:53:06.302589802 +0000 UTC m=+1008.018934873 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls") pod "maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85" (UID: "ffa5c77a-9830-4201-8b01-47edec592238") : secret "maas-api-e2e-api-a-87293f-serving-cert" not found
Oct 30 18:53:04.384649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:04.384552    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": context deadline exceeded"
Oct 30 18:53:04.433125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:04.399401    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:04.706279 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:04.706235    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 18:53:04.719616 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:04.719575    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:04.740218 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:04.733178    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 18:53:05.057177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.056205    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:53:05.057177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.056279    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:05.244216 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.244169    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:05.456036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.447948    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:53:05.601144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.600368    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:05.732336 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.732227    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 18:53:05.740951 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.739854    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:05.831464 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.831364    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:05.884396 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.884343    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f" event={"ID":"25d54e45-58d2-4476-869c-4b076b617038","Type":"ContainerStarted","Data":"7fc17518f248287d41537c3042dae23712b0763814f649edbbc23a8269e81a8c"}
Oct 30 18:53:05.888517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.887165    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:53:05.929117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.926801    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f" podStartSLOduration=5.616064286 podStartE2EDuration="6.926781705s" totalImagesPullingTime="1.310717419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:05.920520773 +0000 UTC m=+1007.636865864" watchObservedRunningTime="2026-10-30 18:53:05.926781705 +0000 UTC m=+1007.643126782"
Oct 30 18:53:05.929117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.929085    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:53:05.967042 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:05.967005    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:53:06.055327 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:06.055244    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:06.354214 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:06.341591    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:06.446554 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:06.446397    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls\") pod \"maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:06.500449 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:06.500417    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:06.751669 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:06.751626    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:07.175940 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:07.175897    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:53:07.263121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:07.263007    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:07.443075 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:07.443046    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"]
Oct 30 18:53:08.239588 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:08.238112    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:08.246175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:08.246047    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85" event={"ID":"ffa5c77a-9830-4201-8b01-47edec592238","Type":"ContainerStarted","Data":"ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7"}
Oct 30 18:53:08.246175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:08.246112    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85" event={"ID":"ffa5c77a-9830-4201-8b01-47edec592238","Type":"ContainerStarted","Data":"316b68275189134f6af3bd464ab903e3e79f8df7aaebd1426856595b787ab150"}
Oct 30 18:53:08.267442 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:08.267385    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85" podStartSLOduration=5.830942931 podStartE2EDuration="6.267370375s" totalImagesPullingTime="436.427444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:08.265285455 +0000 UTC m=+1009.981630528" watchObservedRunningTime="2026-10-30 18:53:08.267370375 +0000 UTC m=+1009.983715463"
Oct 30 18:53:08.789792 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:08.789741    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:09.234149 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:09.229495    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:09.258118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:09.257746    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:09.539939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:09.539856    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"
Oct 30 18:53:09.539939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:09.539924    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"
Oct 30 18:53:09.540137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:09.539963    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"
Oct 30 18:53:10.028119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:10.027636    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:10.737120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:10.734273    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:11.486128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:11.484729    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:12.171162 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:12.166861    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:12.783819 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:12.783706    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:13.251374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:13.251348    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:53:13.303445 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:53:13.303404    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:53:13.443810 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:13.443774    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:15.279644 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:15.279616    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:53:15.573035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:15.572947    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:15.975122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:15.974296    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:16.588296 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:16.588258    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:17.544127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:17.539315    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:18.362120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:18.361700    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:19.065936 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:19.065899    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:19.665145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:19.661595    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:20.128974 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:20.128879    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:20.361151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:20.357598    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:20.580280 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:20.580245    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:20.797417 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:20.797381    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:21.086559 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:21.086525    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:21.295914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:21.295862    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:21.509298 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:21.507028    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:21.874056 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:21.873974    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:22.058028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:22.057990    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:22.298085 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:22.298051    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:23.082128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:23.079559    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:23.411982 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:23.411946    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:23.814797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:23.814693    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:24.477933 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:24.477899    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:25.485123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:25.477772    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:25.891719 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:25.891606    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:26.091141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:26.090986    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:26.806069 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:26.806036    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:27.201125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.200237    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"]
Oct 30 18:53:27.204472 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.204445    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:27.206611 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.206592    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"]
Oct 30 18:53:27.206758 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.206734    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"
Oct 30 18:53:27.211992 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.211951    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"]
Oct 30 18:53:27.216598 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.216575    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"
Oct 30 18:53:27.218683 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.218658    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-066928\""
Oct 30 18:53:27.224697 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.224645    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"]
Oct 30 18:53:27.225990 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.225968    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-066928-dockercfg-k47bn\""
Oct 30 18:53:27.398184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.398136    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s94l4\" (UniqueName: \"kubernetes.io/projected/33d3855a-6e7b-48a4-b842-41d77ea5e317-kube-api-access-s94l4\") pod \"payload-processing-e2e-api-b-066928-5866c665f8-pzkx5\" (UID: \"33d3855a-6e7b-48a4-b842-41d77ea5e317\") " pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"
Oct 30 18:53:27.398654 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.398624    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d3855a-6e7b-48a4-b842-41d77ea5e317-plugins-config-volume\") pod \"payload-processing-e2e-api-b-066928-5866c665f8-pzkx5\" (UID: \"33d3855a-6e7b-48a4-b842-41d77ea5e317\") " pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"
Oct 30 18:53:27.398878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.398857    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2rtr\" (UniqueName: \"kubernetes.io/projected/d1090131-35f1-4572-bb89-bd36236c5c66-kube-api-access-t2rtr\") pod \"payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79\" (UID: \"d1090131-35f1-4572-bb89-bd36236c5c66\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"
Oct 30 18:53:27.399072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.399053    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1090131-35f1-4572-bb89-bd36236c5c66-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79\" (UID: \"d1090131-35f1-4572-bb89-bd36236c5c66\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"
Oct 30 18:53:27.462570 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.462485    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:27.530200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.530160    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2rtr\" (UniqueName: \"kubernetes.io/projected/d1090131-35f1-4572-bb89-bd36236c5c66-kube-api-access-t2rtr\") pod \"payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79\" (UID: \"d1090131-35f1-4572-bb89-bd36236c5c66\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"
Oct 30 18:53:27.578828 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.578059    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1090131-35f1-4572-bb89-bd36236c5c66-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79\" (UID: \"d1090131-35f1-4572-bb89-bd36236c5c66\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"
Oct 30 18:53:27.578828 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.578181    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s94l4\" (UniqueName: \"kubernetes.io/projected/33d3855a-6e7b-48a4-b842-41d77ea5e317-kube-api-access-s94l4\") pod \"payload-processing-e2e-api-b-066928-5866c665f8-pzkx5\" (UID: \"33d3855a-6e7b-48a4-b842-41d77ea5e317\") " pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"
Oct 30 18:53:27.578828 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.578420    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d3855a-6e7b-48a4-b842-41d77ea5e317-plugins-config-volume\") pod \"payload-processing-e2e-api-b-066928-5866c665f8-pzkx5\" (UID: \"33d3855a-6e7b-48a4-b842-41d77ea5e317\") " pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"
Oct 30 18:53:27.580629 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.579411    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d3855a-6e7b-48a4-b842-41d77ea5e317-plugins-config-volume\") pod \"payload-processing-e2e-api-b-066928-5866c665f8-pzkx5\" (UID: \"33d3855a-6e7b-48a4-b842-41d77ea5e317\") " pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"
Oct 30 18:53:27.580629 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.580232    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1090131-35f1-4572-bb89-bd36236c5c66-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79\" (UID: \"d1090131-35f1-4572-bb89-bd36236c5c66\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"
Oct 30 18:53:27.617039 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.617009    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2rtr\" (UniqueName: \"kubernetes.io/projected/d1090131-35f1-4572-bb89-bd36236c5c66-kube-api-access-t2rtr\") pod \"payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79\" (UID: \"d1090131-35f1-4572-bb89-bd36236c5c66\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"
Oct 30 18:53:27.627382 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.627321    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s94l4\" (UniqueName: \"kubernetes.io/projected/33d3855a-6e7b-48a4-b842-41d77ea5e317-kube-api-access-s94l4\") pod \"payload-processing-e2e-api-b-066928-5866c665f8-pzkx5\" (UID: \"33d3855a-6e7b-48a4-b842-41d77ea5e317\") " pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"
Oct 30 18:53:27.834496 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.834409    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"
Oct 30 18:53:27.848929 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:27.848904    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"
Oct 30 18:53:28.098159 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:28.098041    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:28.209904 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:28.209875    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"]
Oct 30 18:53:28.438183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:28.397616    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:28.537707 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:28.537679    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"]
Oct 30 18:53:28.621537 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:28.621505    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79" event={"ID":"d1090131-35f1-4572-bb89-bd36236c5c66","Type":"ContainerStarted","Data":"b1b81aa947a55cc50e7de35992d881c1754561d78c619e7685715badba2d88d7"}
Oct 30 18:53:28.622931 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:28.622912    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5" event={"ID":"33d3855a-6e7b-48a4-b842-41d77ea5e317","Type":"ContainerStarted","Data":"9f422539b32c8a17983eb8d6de62fe8fcf566c315d23eacc3e88e52ebcc6acb5"}
Oct 30 18:53:29.081183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:29.074011    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:29.672121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:29.667944    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79" event={"ID":"d1090131-35f1-4572-bb89-bd36236c5c66","Type":"ContainerStarted","Data":"dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187"}
Oct 30 18:53:29.672121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:29.668909    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"
Oct 30 18:53:29.684122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:29.683359    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5" event={"ID":"33d3855a-6e7b-48a4-b842-41d77ea5e317","Type":"ContainerStarted","Data":"738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9"}
Oct 30 18:53:29.698119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:29.688228    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"
Oct 30 18:53:29.698119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:29.696695    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79" podStartSLOduration=2.696668931 podStartE2EDuration="2.696668931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:29.695968086 +0000 UTC m=+1031.412313163" watchObservedRunningTime="2026-10-30 18:53:29.696668931 +0000 UTC m=+1031.413014010"
Oct 30 18:53:29.799281 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:29.760921    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5" podStartSLOduration=2.760898336 podStartE2EDuration="2.760898336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:29.73279834 +0000 UTC m=+1031.449143420" watchObservedRunningTime="2026-10-30 18:53:29.760898336 +0000 UTC m=+1031.477243413"
Oct 30 18:53:29.990610 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:29.990527    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:30.365190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:30.365104    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:30.762609 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:30.762528    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:31.593118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:31.581846    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:32.173218 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.172068    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:32.717883 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.717851    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"]
Oct 30 18:53:32.738931 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.738904    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"]
Oct 30 18:53:32.739141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.739028    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:32.743144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.743068    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-066928-serving-cert\""
Oct 30 18:53:32.754451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.750511    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q22cn\" (UniqueName: \"kubernetes.io/projected/8101b22c-2203-455a-943b-1372e7427e99-kube-api-access-q22cn\") pod \"maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:32.754451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.750714    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-maas-api-tls\") pod \"maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:32.754451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.750752    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8101b22c-2203-455a-943b-1372e7427e99-openshift-service-ca\") pod \"maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:32.754451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.750779    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-metrics-certs\") pod \"maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:32.865121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.854128    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-maas-api-tls\") pod \"maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:32.865121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.858011    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:32.894224 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.894186    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8101b22c-2203-455a-943b-1372e7427e99-openshift-service-ca\") pod \"maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:32.894543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.854181    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8101b22c-2203-455a-943b-1372e7427e99-openshift-service-ca\") pod \"maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:32.894729 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.894710    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-metrics-certs\") pod \"maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:32.895029 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.894982    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q22cn\" (UniqueName: \"kubernetes.io/projected/8101b22c-2203-455a-943b-1372e7427e99-kube-api-access-q22cn\") pod \"maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:32.909781 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.909750    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-metrics-certs\") pod \"maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:32.918191 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.918127    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-maas-api-tls\") pod \"maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:32.973611 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:32.973445    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q22cn\" (UniqueName: \"kubernetes.io/projected/8101b22c-2203-455a-943b-1372e7427e99-kube-api-access-q22cn\") pod \"maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:33.042409 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.042372    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:33.066864 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.066834    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:33.409214 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.404550    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"]
Oct 30 18:53:33.521929 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.521891    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"]
Oct 30 18:53:33.529138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.527335    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.543673 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.543649    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-7784e5-serving-cert\""
Oct 30 18:53:33.550526 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.550500    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"]
Oct 30 18:53:33.644269 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.644187    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-maas-api-tls\") pod \"maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.644441 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.644295    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-metrics-certs\") pod \"maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.644441 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.644375    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l77g9\" (UniqueName: \"kubernetes.io/projected/f0b83aec-e567-4e29-b17b-3f5c697982e9-kube-api-access-l77g9\") pod \"maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.644441 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.644397    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0b83aec-e567-4e29-b17b-3f5c697982e9-openshift-service-ca\") pod \"maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.652449 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.652420    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:33.808121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.797781    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv" event={"ID":"8101b22c-2203-455a-943b-1372e7427e99","Type":"ContainerStarted","Data":"545ffdad01c24c555e1ebb0de9d065ea23a1b004864ebd41dbab496e5f27e586"}
Oct 30 18:53:33.808121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.799510    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-metrics-certs\") pod \"maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.808121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.799635    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l77g9\" (UniqueName: \"kubernetes.io/projected/f0b83aec-e567-4e29-b17b-3f5c697982e9-kube-api-access-l77g9\") pod \"maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.808121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.799673    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0b83aec-e567-4e29-b17b-3f5c697982e9-openshift-service-ca\") pod \"maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.808121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.799878    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-maas-api-tls\") pod \"maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.808121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.801617    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0b83aec-e567-4e29-b17b-3f5c697982e9-openshift-service-ca\") pod \"maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.821165 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.811957    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-metrics-certs\") pod \"maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.821165 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.814716    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-maas-api-tls\") pod \"maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.821165 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.815938    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l77g9\" (UniqueName: \"kubernetes.io/projected/f0b83aec-e567-4e29-b17b-3f5c697982e9-kube-api-access-l77g9\") pod \"maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:33.838036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.838002    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:33.856013 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:33.855989    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:34.241146 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:34.241084    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"]
Oct 30 18:53:34.630179 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:34.622287    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:34.808882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:34.808767    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9" event={"ID":"f0b83aec-e567-4e29-b17b-3f5c697982e9","Type":"ContainerStarted","Data":"f512bf4c4b824dae4dbcb05bf53b1b434963fc983641804376f4705ead414314"}
Oct 30 18:53:34.808882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:34.808824    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9" event={"ID":"f0b83aec-e567-4e29-b17b-3f5c697982e9","Type":"ContainerStarted","Data":"ca56c2f1d6b10e1fc09d36a7cf62a2225fd0aee604792212a1f45bfa4bc32b13"}
Oct 30 18:53:34.810300 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:34.808889    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:34.810638 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:34.810615    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv" event={"ID":"8101b22c-2203-455a-943b-1372e7427e99","Type":"ContainerStarted","Data":"c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563"}
Oct 30 18:53:34.810714 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:34.810687    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:34.830349 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:34.830307    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9" podStartSLOduration=1.403675906 podStartE2EDuration="1.830293594s" totalImagesPullingTime="426.617688ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:34.828267694 +0000 UTC m=+1036.544612774" watchObservedRunningTime="2026-10-30 18:53:34.830293594 +0000 UTC m=+1036.546638669"
Oct 30 18:53:34.851643 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:34.851588    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv" podStartSLOduration=2.269962326 podStartE2EDuration="2.851569834s" totalImagesPullingTime="581.607508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:34.845897257 +0000 UTC m=+1036.562242333" watchObservedRunningTime="2026-10-30 18:53:34.851569834 +0000 UTC m=+1036.567914913"
Oct 30 18:53:35.534012 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:35.533974    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:35.822562 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:35.822472    4925 generic.go:356] "Generic (PLEG): container finished" podID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerID="4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c" exitCode=0
Oct 30 18:53:35.822562 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:35.822546    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" event={"ID":"4e752814-e14b-43f0-9401-71f46afb9a6b","Type":"ContainerDied","Data":"4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c"}
Oct 30 18:53:35.823481 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:35.823456    4925 scope.go:122] "RemoveContainer" containerID="4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c"
Oct 30 18:53:35.825912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:35.825869    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:35.826029 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:35.825960    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:35.826029 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:35.826015    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:36.373121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:36.370983    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:36.833920 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:36.833885    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" event={"ID":"4e752814-e14b-43f0-9401-71f46afb9a6b","Type":"ContainerStarted","Data":"56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55"}
Oct 30 18:53:37.063370 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:37.062132    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:37.063370 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:37.062472    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:53:37.073123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:37.066254    4925 patch_prober.go:29] interesting pod/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 18:53:37.073123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:37.066310    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 18:53:37.746122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:37.737109    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:38.080167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:38.072445    4925 patch_prober.go:29] interesting pod/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 18:53:38.080167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:38.072510    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 18:53:38.336739 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:38.336637    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:53:38.491124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:38.489323    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:39.073121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:39.070465    4925 patch_prober.go:29] interesting pod/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 18:53:39.073121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:39.070541    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 18:53:39.258530 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:39.258492    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:39.493732 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:39.493698    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:39.760385 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:39.728668    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:40.079184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:40.070575    4925 patch_prober.go:29] interesting pod/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 18:53:40.079184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:40.070638    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 18:53:40.134165 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:40.134130    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:40.316120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:40.314247    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:40.829594 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:40.829562    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:53:40.829898 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:40.829873    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:53:41.132184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:41.114831    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:41.132184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:41.115991    4925 patch_prober.go:29] interesting pod/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 18:53:41.132184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:41.116054    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 18:53:41.704069 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:41.704045    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"
Oct 30 18:53:41.704576 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:41.704561    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"
Oct 30 18:53:42.097179 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:42.081871    4925 patch_prober.go:29] interesting pod/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 30 18:53:42.123180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:42.121204    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 30 18:53:42.342507 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:42.342469    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:43.089041 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:43.087425    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:53:43.092473 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:43.092450    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:53:43.317835 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:53:43.317798    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:53:43.456527 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:43.456480    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:43.680122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:43.678187    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:44.291640 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.291605    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:44.604147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.604054    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"]
Oct 30 18:53:44.607786 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.607763    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"
Oct 30 18:53:44.616078 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.616058    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-da58b769\""
Oct 30 18:53:44.619866 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.619839    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"]
Oct 30 18:53:44.659843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.659809    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:44.764980 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.764933    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frvdq\" (UniqueName: \"kubernetes.io/projected/ed2c0474-08cf-47f7-a431-a35864c221d2-kube-api-access-frvdq\") pod \"payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj\" (UID: \"ed2c0474-08cf-47f7-a431-a35864c221d2\") " pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"
Oct 30 18:53:44.765374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.765352    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed2c0474-08cf-47f7-a431-a35864c221d2-plugins-config-volume\") pod \"payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj\" (UID: \"ed2c0474-08cf-47f7-a431-a35864c221d2\") " pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"
Oct 30 18:53:44.848884 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.848851    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:44.870899 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.869458    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frvdq\" (UniqueName: \"kubernetes.io/projected/ed2c0474-08cf-47f7-a431-a35864c221d2-kube-api-access-frvdq\") pod \"payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj\" (UID: \"ed2c0474-08cf-47f7-a431-a35864c221d2\") " pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"
Oct 30 18:53:44.870899 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.869504    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed2c0474-08cf-47f7-a431-a35864c221d2-plugins-config-volume\") pod \"payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj\" (UID: \"ed2c0474-08cf-47f7-a431-a35864c221d2\") " pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"
Oct 30 18:53:44.870899 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.870491    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed2c0474-08cf-47f7-a431-a35864c221d2-plugins-config-volume\") pod \"payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj\" (UID: \"ed2c0474-08cf-47f7-a431-a35864c221d2\") " pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"
Oct 30 18:53:44.920460 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.920428    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frvdq\" (UniqueName: \"kubernetes.io/projected/ed2c0474-08cf-47f7-a431-a35864c221d2-kube-api-access-frvdq\") pod \"payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj\" (UID: \"ed2c0474-08cf-47f7-a431-a35864c221d2\") " pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"
Oct 30 18:53:44.921246 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:44.921227    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"
Oct 30 18:53:45.292163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:45.292126    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"]
Oct 30 18:53:45.297314 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:53:45.297271    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded2c0474_08cf_47f7_a431_a35864c221d2.slice/crio-2a7211b68cdfab4360cd12995f46a898667da074935cffbc52a7574efaf221fd WatchSource:0}: Error finding container 2a7211b68cdfab4360cd12995f46a898667da074935cffbc52a7574efaf221fd: Status 404 returned error can't find the container with id 2a7211b68cdfab4360cd12995f46a898667da074935cffbc52a7574efaf221fd
Oct 30 18:53:45.682927 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:45.682891    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:45.952818 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:45.952748    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:46.163881 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:46.161980    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:46.190361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:46.190330    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj" event={"ID":"ed2c0474-08cf-47f7-a431-a35864c221d2","Type":"ContainerStarted","Data":"88f8210f6788c8106a0aa3c33b364abd447a073691c3288228dce2ffaf5a81a9"}
Oct 30 18:53:46.190597 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:46.190578    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj" event={"ID":"ed2c0474-08cf-47f7-a431-a35864c221d2","Type":"ContainerStarted","Data":"2a7211b68cdfab4360cd12995f46a898667da074935cffbc52a7574efaf221fd"}
Oct 30 18:53:46.190727 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:46.190717    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"
Oct 30 18:53:46.214180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:46.214055    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj" podStartSLOduration=2.214038879 podStartE2EDuration="2.214038879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:46.212292758 +0000 UTC m=+1047.928637839" watchObservedRunningTime="2026-10-30 18:53:46.214038879 +0000 UTC m=+1047.930383957"
Oct 30 18:53:47.183999 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:47.183921    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:48.049087 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:48.049056    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:49.048771 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:49.048728    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:50.033122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:50.031776    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:50.970122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:50.963750    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:51.506959 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.506928    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"]
Oct 30 18:53:51.512201 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.512170    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:51.535454 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.535407    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-da58b769-serving-cert\""
Oct 30 18:53:51.538556 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.538529    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"]
Oct 30 18:53:51.672124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.666003    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-maas-api-tls\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:51.672124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.666121    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-metrics-certs\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:51.672124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.666402    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9glw\" (UniqueName: \"kubernetes.io/projected/a8545208-0e08-4181-9dc6-b498a6ed0ae1-kube-api-access-w9glw\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:51.672124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.666412    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:51.672124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.666453    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8545208-0e08-4181-9dc6-b498a6ed0ae1-openshift-service-ca\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:51.767736 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.767658    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-maas-api-tls\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:51.768075 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:53:51.768059    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-serving-cert: secret "maas-api-e2e-mt-da58b769-serving-cert" not found
Oct 30 18:53:51.768316 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:53:51.768307    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-maas-api-tls podName:a8545208-0e08-4181-9dc6-b498a6ed0ae1 nodeName:}" failed. No retries permitted until 2026-10-30 18:53:52.26828979 +0000 UTC m=+1053.984634856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-maas-api-tls") pod "maas-api-e2e-mt-da58b769-c58747d4f-9phxv" (UID: "a8545208-0e08-4181-9dc6-b498a6ed0ae1") : secret "maas-api-e2e-mt-da58b769-serving-cert" not found
Oct 30 18:53:51.768924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.768181    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-metrics-certs\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:51.769311 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.769284    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9glw\" (UniqueName: \"kubernetes.io/projected/a8545208-0e08-4181-9dc6-b498a6ed0ae1-kube-api-access-w9glw\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:51.769447 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.769435    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8545208-0e08-4181-9dc6-b498a6ed0ae1-openshift-service-ca\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:51.770507 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.770482    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8545208-0e08-4181-9dc6-b498a6ed0ae1-openshift-service-ca\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:51.773369 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.773345    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-metrics-certs\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:51.779264 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.779240    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9glw\" (UniqueName: \"kubernetes.io/projected/a8545208-0e08-4181-9dc6-b498a6ed0ae1-kube-api-access-w9glw\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:51.839308 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:51.839274    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:52.016148 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:52.016018    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:52.294574 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:52.294538    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-maas-api-tls\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:52.297415 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:52.297378    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-maas-api-tls\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-9phxv\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:52.453462 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:52.453427    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:52.497814 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:52.497734    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:52.813873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:52.813839    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"]
Oct 30 18:53:52.815243 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:53:52.815217    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8545208_0e08_4181_9dc6_b498a6ed0ae1.slice/crio-c06306d598378a16c1ae3994772fa90302b73b3a5e5ed270c8a41ca3ee566f36 WatchSource:0}: Error finding container c06306d598378a16c1ae3994772fa90302b73b3a5e5ed270c8a41ca3ee566f36: Status 404 returned error can't find the container with id c06306d598378a16c1ae3994772fa90302b73b3a5e5ed270c8a41ca3ee566f36
Oct 30 18:53:53.281920 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:53.281669    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:53.290298 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:53.290234    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv" event={"ID":"a8545208-0e08-4181-9dc6-b498a6ed0ae1","Type":"ContainerStarted","Data":"c06306d598378a16c1ae3994772fa90302b73b3a5e5ed270c8a41ca3ee566f36"}
Oct 30 18:53:54.033147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:54.025215    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:54.299230 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:54.299128    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv" event={"ID":"a8545208-0e08-4181-9dc6-b498a6ed0ae1","Type":"ContainerStarted","Data":"54def9e6448efc7bb3d406db8ea26e07a70466b19a45682db3b90529de519409"}
Oct 30 18:53:54.299710 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:54.299275    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:53:54.319713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:54.319658    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv" podStartSLOduration=2.844648014 podStartE2EDuration="3.319646109s" totalImagesPullingTime="474.998095ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:54.316367312 +0000 UTC m=+1056.032712387" watchObservedRunningTime="2026-10-30 18:53:54.319646109 +0000 UTC m=+1056.035991185"
Oct 30 18:53:54.864767 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:54.864735    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:55.662123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:55.659126    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:56.260472 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:56.259644    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:56.947446 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:56.947409    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"]
Oct 30 18:53:57.062066 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:57.061889    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:57.209305 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:57.209219    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"
Oct 30 18:53:57.250490 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:57.250140    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:57.868605 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:57.868575    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:58.343960 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:58.343925    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"]
Oct 30 18:53:58.363014 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:58.362976    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"]
Oct 30 18:53:58.506790 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:58.506752    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:58.693716 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:58.693680    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:59.381117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:59.379231    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:59.976074 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:53:59.976039    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:00.313346 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:00.313253    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:54:00.658981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:00.655189    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:01.327207 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:01.327160    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:01.959126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:01.952554    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn" podUID="8e205f84-0fcf-4ed1-9f98-a8672dcb29eb" containerName="payload-processing" containerID="cri-o://002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904" gracePeriod=25
Oct 30 18:54:01.980267 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:01.980231    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:02.272493 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.272464    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"
Oct 30 18:54:02.400574 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.400537    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-plugins-config-volume\") pod \"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb\" (UID: \"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb\") "
Oct 30 18:54:02.400782 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.400684    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-kube-api-access-glm8d\" (UniqueName: \"kubernetes.io/projected/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-kube-api-access-glm8d\") pod \"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb\" (UID: \"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb\") "
Oct 30 18:54:02.401024 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.400997    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-plugins-config-volume" pod "8e205f84-0fcf-4ed1-9f98-a8672dcb29eb" (UID: "8e205f84-0fcf-4ed1-9f98-a8672dcb29eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:02.401189 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.401169    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:02.402926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.402903    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-kube-api-access-glm8d" pod "8e205f84-0fcf-4ed1-9f98-a8672dcb29eb" (UID: "8e205f84-0fcf-4ed1-9f98-a8672dcb29eb"). InnerVolumeSpecName "kube-api-access-glm8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:02.502471 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.502390    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glm8d\" (UniqueName: \"kubernetes.io/projected/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb-kube-api-access-glm8d\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:02.584868 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.584831    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:02.594130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.593341    4925 generic.go:356] "Generic (PLEG): container finished" podID="8e205f84-0fcf-4ed1-9f98-a8672dcb29eb" containerID="002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904" exitCode=0
Oct 30 18:54:02.594130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.593604    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn" event={"ID":"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb","Type":"ContainerDied","Data":"002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904"}
Oct 30 18:54:02.594130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.593652    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn" event={"ID":"8e205f84-0fcf-4ed1-9f98-a8672dcb29eb","Type":"ContainerDied","Data":"36870be960dd7a0994b8cbb1ff0f46eb6dd878665d328f4d0150cdb78fadec04"}
Oct 30 18:54:02.594130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.593680    4925 scope.go:122] "RemoveContainer" containerID="002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904"
Oct 30 18:54:02.594130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.593829    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"
Oct 30 18:54:02.642621 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.642593    4925 scope.go:122] "RemoveContainer" containerID="002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904"
Oct 30 18:54:02.643236 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:54:02.643206    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904\": container with ID starting with 002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904 not found: ID does not exist" containerID="002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904"
Oct 30 18:54:02.643362 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.643245    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904"} err="failed to get container status \"002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904\": rpc error: code = NotFound desc = could not find container \"002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904\": container with ID starting with 002b03920458fdb7fa0fd27560303a5739ddb4983a462fd6509e6729e5a4f904 not found: ID does not exist"
Oct 30 18:54:02.645640 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.645623    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"]
Oct 30 18:54:02.648947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:02.648924    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-87293f-68d6999dd8-cv2vn"]
Oct 30 18:54:03.293122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.290495    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:03.345030 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.344974    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d" podUID="9a7ef085-d27e-4a4c-8ca7-ff28e7175777" containerName="payload-processing" containerID="cri-o://407125055fe39c3f587c44bee6db6b1aee97880352961469c2e858c3600ec05a" gracePeriod=25
Oct 30 18:54:03.365220 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.365179    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4" podUID="eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5" containerName="payload-pre-processing" containerID="cri-o://b5b76adafc084a4c561fcbeeaf6a8154cf1ffbab4e81ede41337dc55be106ddc" gracePeriod=25
Oct 30 18:54:03.604453 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.604282    4925 generic.go:356] "Generic (PLEG): container finished" podID="eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5" containerID="b5b76adafc084a4c561fcbeeaf6a8154cf1ffbab4e81ede41337dc55be106ddc" exitCode=0
Oct 30 18:54:03.604453 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.604380    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4" event={"ID":"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5","Type":"ContainerDied","Data":"b5b76adafc084a4c561fcbeeaf6a8154cf1ffbab4e81ede41337dc55be106ddc"}
Oct 30 18:54:03.608785 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.608639    4925 generic.go:356] "Generic (PLEG): container finished" podID="9a7ef085-d27e-4a4c-8ca7-ff28e7175777" containerID="407125055fe39c3f587c44bee6db6b1aee97880352961469c2e858c3600ec05a" exitCode=0
Oct 30 18:54:03.608785 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.608732    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d" event={"ID":"9a7ef085-d27e-4a4c-8ca7-ff28e7175777","Type":"ContainerDied","Data":"407125055fe39c3f587c44bee6db6b1aee97880352961469c2e858c3600ec05a"}
Oct 30 18:54:03.747417 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.747387    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"
Oct 30 18:54:03.763413 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.763391    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"
Oct 30 18:54:03.838717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.838623    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-plugins-config-volume\") pod \"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5\" (UID: \"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5\") "
Oct 30 18:54:03.838893 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.838756    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-plugins-config-volume\") pod \"9a7ef085-d27e-4a4c-8ca7-ff28e7175777\" (UID: \"9a7ef085-d27e-4a4c-8ca7-ff28e7175777\") "
Oct 30 18:54:03.838893 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.838844    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-kube-api-access-s6r2l\" (UniqueName: \"kubernetes.io/projected/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-kube-api-access-s6r2l\") pod \"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5\" (UID: \"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5\") "
Oct 30 18:54:03.839018 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.838931    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-kube-api-access-dlmh4\" (UniqueName: \"kubernetes.io/projected/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-kube-api-access-dlmh4\") pod \"9a7ef085-d27e-4a4c-8ca7-ff28e7175777\" (UID: \"9a7ef085-d27e-4a4c-8ca7-ff28e7175777\") "
Oct 30 18:54:03.839159 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.839126    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-plugins-config-volume" pod "eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5" (UID: "eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:03.839561 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.839528    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-plugins-config-volume" pod "9a7ef085-d27e-4a4c-8ca7-ff28e7175777" (UID: "9a7ef085-d27e-4a4c-8ca7-ff28e7175777"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:03.839702 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.839682    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:03.839836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.839708    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:03.842180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.842149    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-kube-api-access-dlmh4" pod "9a7ef085-d27e-4a4c-8ca7-ff28e7175777" (UID: "9a7ef085-d27e-4a4c-8ca7-ff28e7175777"). InnerVolumeSpecName "kube-api-access-dlmh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:03.842290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.842201    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-kube-api-access-s6r2l" pod "eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5" (UID: "eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5"). InnerVolumeSpecName "kube-api-access-s6r2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:03.941553 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.941520    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6r2l\" (UniqueName: \"kubernetes.io/projected/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5-kube-api-access-s6r2l\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:03.941553 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:03.941549    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlmh4\" (UniqueName: \"kubernetes.io/projected/9a7ef085-d27e-4a4c-8ca7-ff28e7175777-kube-api-access-dlmh4\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:04.162179 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.161590    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:04.424709 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.424636    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e205f84-0fcf-4ed1-9f98-a8672dcb29eb" path="/var/lib/kubelet/pods/8e205f84-0fcf-4ed1-9f98-a8672dcb29eb/volumes"
Oct 30 18:54:04.617266 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.617235    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"
Oct 30 18:54:04.617266 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.617251    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d" event={"ID":"9a7ef085-d27e-4a4c-8ca7-ff28e7175777","Type":"ContainerDied","Data":"96c1b05d9a85679ab0c3b6694429123693e43241c637e712136cad97c182fe1b"}
Oct 30 18:54:04.617510 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.617334    4925 scope.go:122] "RemoveContainer" containerID="407125055fe39c3f587c44bee6db6b1aee97880352961469c2e858c3600ec05a"
Oct 30 18:54:04.619074 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.619054    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"
Oct 30 18:54:04.619219 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.619054    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4" event={"ID":"eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5","Type":"ContainerDied","Data":"34da69742c7088cebfa2437ae7e34bc7611d33257ab2f317ff670ffbcca4d14d"}
Oct 30 18:54:04.628184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.628076    4925 scope.go:122] "RemoveContainer" containerID="b5b76adafc084a4c561fcbeeaf6a8154cf1ffbab4e81ede41337dc55be106ddc"
Oct 30 18:54:04.639886 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.639713    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"]
Oct 30 18:54:04.643374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.643356    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f37d42-6b68688f7f-tx6m4"]
Oct 30 18:54:04.653035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.653006    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"]
Oct 30 18:54:04.655705 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.655688    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f37d42-78585ddd78-qlc7d"]
Oct 30 18:54:04.844043 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:04.843804    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:05.430924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:05.430892    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:06.296342 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:06.296289    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:06.423633 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:06.423602    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a7ef085-d27e-4a4c-8ca7-ff28e7175777" path="/var/lib/kubelet/pods/9a7ef085-d27e-4a4c-8ca7-ff28e7175777/volumes"
Oct 30 18:54:06.424090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:06.424073    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5" path="/var/lib/kubelet/pods/eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5/volumes"
Oct 30 18:54:06.993984 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:06.989824    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:07.776730 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:07.776697    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:08.482465 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:08.482425    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:08.695771 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:08.695739    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:08.881949 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:08.881856    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:09.306122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:09.302518    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:09.940500 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:09.940470    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"]
Oct 30 18:54:09.940980 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:09.940728    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85" podUID="ffa5c77a-9830-4201-8b01-47edec592238" containerName="maas-api" containerID="cri-o://ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7" gracePeriod=30
Oct 30 18:54:10.071991 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.071866    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:10.409191 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.393220    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:10.550229 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.549693    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"]
Oct 30 18:54:10.550878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.550509    4925 cpu_manager.go:414] "containerMap: removing container" podUID="9a7ef085-d27e-4a4c-8ca7-ff28e7175777" containerName="payload-processing"
Oct 30 18:54:10.550878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.550531    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a7ef085-d27e-4a4c-8ca7-ff28e7175777" containerName="payload-processing"
Oct 30 18:54:10.550878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.550568    4925 cpu_manager.go:414] "containerMap: removing container" podUID="8e205f84-0fcf-4ed1-9f98-a8672dcb29eb" containerName="payload-processing"
Oct 30 18:54:10.550878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.550576    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e205f84-0fcf-4ed1-9f98-a8672dcb29eb" containerName="payload-processing"
Oct 30 18:54:10.550878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.550599    4925 cpu_manager.go:414] "containerMap: removing container" podUID="eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5" containerName="payload-pre-processing"
Oct 30 18:54:10.550878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.550607    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5" containerName="payload-pre-processing"
Oct 30 18:54:10.550878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.550774    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a7ef085-d27e-4a4c-8ca7-ff28e7175777" containerName="payload-processing"
Oct 30 18:54:10.550878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.550787    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e205f84-0fcf-4ed1-9f98-a8672dcb29eb" containerName="payload-processing"
Oct 30 18:54:10.550878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.550799    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="eaf7a6bc-5f1e-42b3-ab9c-da01dde9cdd5" containerName="payload-pre-processing"
Oct 30 18:54:10.554118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.554072    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"]
Oct 30 18:54:10.554212 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.554192    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.558981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.558943    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-da58b769\"/\"kube-root-ca.crt\""
Oct 30 18:54:10.559119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.559040    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-da58b769\"/\"e2e-lifecycle-model-da58b769-kserve-self-signed-certs\""
Oct 30 18:54:10.559613 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.559588    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-da58b769\"/\"default-dockercfg-d2nkh\""
Oct 30 18:54:10.559721 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.559656    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-da58b769\"/\"openshift-service-ca.crt\""
Oct 30 18:54:10.584537 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.584501    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:10.598213 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.598190    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:54:10.684091 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.684002    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffa5c77a-9830-4201-8b01-47edec592238-kube-api-access-b22xf\" (UniqueName: \"kubernetes.io/projected/ffa5c77a-9830-4201-8b01-47edec592238-kube-api-access-b22xf\") pod \"ffa5c77a-9830-4201-8b01-47edec592238\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") "
Oct 30 18:54:10.684355 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.684332    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffa5c77a-9830-4201-8b01-47edec592238-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffa5c77a-9830-4201-8b01-47edec592238-openshift-service-ca\") pod \"ffa5c77a-9830-4201-8b01-47edec592238\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") "
Oct 30 18:54:10.684447 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.684426    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-metrics-certs\") pod \"ffa5c77a-9830-4201-8b01-47edec592238\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") "
Oct 30 18:54:10.684521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.684501    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls\") pod \"ffa5c77a-9830-4201-8b01-47edec592238\" (UID: \"ffa5c77a-9830-4201-8b01-47edec592238\") "
Oct 30 18:54:10.685003 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.684979    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qwbg\" (UniqueName: \"kubernetes.io/projected/89e0f65a-5731-4b10-ae2b-19b559064edb-kube-api-access-7qwbg\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.685071 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.685053    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-model-cache\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.685165 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.685108    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-home\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.685225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.685165    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-tmp-dir\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.685225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.685191    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e0f65a-5731-4b10-ae2b-19b559064edb-tls-certs\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.685324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.685230    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-dshm\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.686793 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.686760    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffa5c77a-9830-4201-8b01-47edec592238-openshift-service-ca" pod "ffa5c77a-9830-4201-8b01-47edec592238" (UID: "ffa5c77a-9830-4201-8b01-47edec592238"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:10.689460 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.689422    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-metrics-certs" pod "ffa5c77a-9830-4201-8b01-47edec592238" (UID: "ffa5c77a-9830-4201-8b01-47edec592238"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:10.695597 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.695574    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffa5c77a-9830-4201-8b01-47edec592238-kube-api-access-b22xf" pod "ffa5c77a-9830-4201-8b01-47edec592238" (UID: "ffa5c77a-9830-4201-8b01-47edec592238"). InnerVolumeSpecName "kube-api-access-b22xf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:10.699997 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.699970    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls" pod "ffa5c77a-9830-4201-8b01-47edec592238" (UID: "ffa5c77a-9830-4201-8b01-47edec592238"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:10.719019 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.718991    4925 generic.go:356] "Generic (PLEG): container finished" podID="ffa5c77a-9830-4201-8b01-47edec592238" containerID="ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7" exitCode=0
Oct 30 18:54:10.719361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.719061    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"
Oct 30 18:54:10.719361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.719088    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85" event={"ID":"ffa5c77a-9830-4201-8b01-47edec592238","Type":"ContainerDied","Data":"ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7"}
Oct 30 18:54:10.719361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.719176    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85" event={"ID":"ffa5c77a-9830-4201-8b01-47edec592238","Type":"ContainerDied","Data":"316b68275189134f6af3bd464ab903e3e79f8df7aaebd1426856595b787ab150"}
Oct 30 18:54:10.719361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.719210    4925 scope.go:122] "RemoveContainer" containerID="ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7"
Oct 30 18:54:10.736253 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.734602    4925 scope.go:122] "RemoveContainer" containerID="ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7"
Oct 30 18:54:10.736253 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:54:10.735005    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7\": container with ID starting with ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7 not found: ID does not exist" containerID="ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7"
Oct 30 18:54:10.736253 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.735040    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7"} err="failed to get container status \"ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7\": rpc error: code = NotFound desc = could not find container \"ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7\": container with ID starting with ce2f5cd7dd93534537aba11fb4d3e31b76604d0796b21a68b2ee0b7b1182b0d7 not found: ID does not exist"
Oct 30 18:54:10.771344 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.771257    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:10.786183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.781035    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"]
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.792065    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qwbg\" (UniqueName: \"kubernetes.io/projected/89e0f65a-5731-4b10-ae2b-19b559064edb-kube-api-access-7qwbg\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.792161    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-model-cache\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.792196    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-home\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.792263    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-tmp-dir\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.792300    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e0f65a-5731-4b10-ae2b-19b559064edb-tls-certs\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.792361    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-dshm\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.792497    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b22xf\" (UniqueName: \"kubernetes.io/projected/ffa5c77a-9830-4201-8b01-47edec592238-kube-api-access-b22xf\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.792514    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffa5c77a-9830-4201-8b01-47edec592238-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.792530    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.792546    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ffa5c77a-9830-4201-8b01-47edec592238-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.793287    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-home\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.793822    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-model-cache\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.794030    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-tmp-dir\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.798912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.797396    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e0f65a-5731-4b10-ae2b-19b559064edb-tls-certs\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.802351 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.802294    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85"]
Oct 30 18:54:10.821124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.818951    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qwbg\" (UniqueName: \"kubernetes.io/projected/89e0f65a-5731-4b10-ae2b-19b559064edb-kube-api-access-7qwbg\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.840118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.835685    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-dshm\") pod \"e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") " pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.869121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.868387    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:10.981166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:10.981075    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:11.229944 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.229913    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"]
Oct 30 18:54:11.231428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:54:11.231354    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89e0f65a_5731_4b10_ae2b_19b559064edb.slice/crio-5c3ac632d44d168ff08210689dccbfaa52692e9d20625fe7bb0f27ec9a09b02e WatchSource:0}: Error finding container 5c3ac632d44d168ff08210689dccbfaa52692e9d20625fe7bb0f27ec9a09b02e: Status 404 returned error can't find the container with id 5c3ac632d44d168ff08210689dccbfaa52692e9d20625fe7bb0f27ec9a09b02e
Oct 30 18:54:11.299485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.299450    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"]
Oct 30 18:54:11.299714 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.299679    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f" podUID="25d54e45-58d2-4476-869c-4b076b617038" containerName="maas-api" containerID="cri-o://7fc17518f248287d41537c3042dae23712b0763814f649edbbc23a8269e81a8c" gracePeriod=30
Oct 30 18:54:11.435758 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.435731    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:11.627973 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.627857    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:11.831180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.818852    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp" event={"ID":"89e0f65a-5731-4b10-ae2b-19b559064edb","Type":"ContainerStarted","Data":"24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831"}
Oct 30 18:54:11.831180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.818900    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp" event={"ID":"89e0f65a-5731-4b10-ae2b-19b559064edb","Type":"ContainerStarted","Data":"5c3ac632d44d168ff08210689dccbfaa52692e9d20625fe7bb0f27ec9a09b02e"}
Oct 30 18:54:11.831180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.824949    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:11.831180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.830602    4925 generic.go:356] "Generic (PLEG): container finished" podID="25d54e45-58d2-4476-869c-4b076b617038" containerID="7fc17518f248287d41537c3042dae23712b0763814f649edbbc23a8269e81a8c" exitCode=0
Oct 30 18:54:11.831180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.830642    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f" event={"ID":"25d54e45-58d2-4476-869c-4b076b617038","Type":"ContainerDied","Data":"7fc17518f248287d41537c3042dae23712b0763814f649edbbc23a8269e81a8c"}
Oct 30 18:54:11.840138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.839336    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:54:11.851224 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.851117    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp" podStartSLOduration=1.851075555 podStartE2EDuration="1.851075555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:11.847992269 +0000 UTC m=+1073.564337347" watchObservedRunningTime="2026-10-30 18:54:11.851075555 +0000 UTC m=+1073.567420632"
Oct 30 18:54:11.869085 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.869055    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:11.950638 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.950609    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25d54e45-58d2-4476-869c-4b076b617038-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25d54e45-58d2-4476-869c-4b076b617038-openshift-service-ca\") pod \"25d54e45-58d2-4476-869c-4b076b617038\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") "
Oct 30 18:54:11.950836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.950700    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25d54e45-58d2-4476-869c-4b076b617038-kube-api-access-p7cdv\" (UniqueName: \"kubernetes.io/projected/25d54e45-58d2-4476-869c-4b076b617038-kube-api-access-p7cdv\") pod \"25d54e45-58d2-4476-869c-4b076b617038\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") "
Oct 30 18:54:11.950836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.950729    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-maas-api-tls\") pod \"25d54e45-58d2-4476-869c-4b076b617038\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") "
Oct 30 18:54:11.950949 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.950835    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-metrics-certs\" (UniqueName: \"kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-metrics-certs\") pod \"25d54e45-58d2-4476-869c-4b076b617038\" (UID: \"25d54e45-58d2-4476-869c-4b076b617038\") "
Oct 30 18:54:11.950949 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.950923    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25d54e45-58d2-4476-869c-4b076b617038-openshift-service-ca" pod "25d54e45-58d2-4476-869c-4b076b617038" (UID: "25d54e45-58d2-4476-869c-4b076b617038"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:11.951687 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.951659    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25d54e45-58d2-4476-869c-4b076b617038-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:11.953133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.953090    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-metrics-certs" pod "25d54e45-58d2-4476-869c-4b076b617038" (UID: "25d54e45-58d2-4476-869c-4b076b617038"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:11.953250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.953221    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25d54e45-58d2-4476-869c-4b076b617038-kube-api-access-p7cdv" pod "25d54e45-58d2-4476-869c-4b076b617038" (UID: "25d54e45-58d2-4476-869c-4b076b617038"). InnerVolumeSpecName "kube-api-access-p7cdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:11.953298 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:11.953252    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-maas-api-tls" pod "25d54e45-58d2-4476-869c-4b076b617038" (UID: "25d54e45-58d2-4476-869c-4b076b617038"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:12.024907 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:12.024681    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:12.054733 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:12.054405    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7cdv\" (UniqueName: \"kubernetes.io/projected/25d54e45-58d2-4476-869c-4b076b617038-kube-api-access-p7cdv\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:12.054733 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:12.054449    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:12.054733 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:12.054465    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25d54e45-58d2-4476-869c-4b076b617038-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:12.327036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:12.326947    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:12.433635 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:12.433598    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffa5c77a-9830-4201-8b01-47edec592238" path="/var/lib/kubelet/pods/ffa5c77a-9830-4201-8b01-47edec592238/volumes"
Oct 30 18:54:12.856924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:12.842288    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"
Oct 30 18:54:12.856924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:12.849394    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f" event={"ID":"25d54e45-58d2-4476-869c-4b076b617038","Type":"ContainerDied","Data":"a6cd493f2fa828ba547cb6ef82bc52ecb6b3539bbca706fd422555c58d938ab3"}
Oct 30 18:54:12.856924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:12.849606    4925 scope.go:122] "RemoveContainer" containerID="7fc17518f248287d41537c3042dae23712b0763814f649edbbc23a8269e81a8c"
Oct 30 18:54:12.910223 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:12.904498    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"]
Oct 30 18:54:12.910223 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:12.910212    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:12.917126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:12.914775    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f37d42-66c757c9db-t7w7f"]
Oct 30 18:54:13.277536 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:13.277498    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-87293f-678d4c9cd5-wjg85" podUID="ffa5c77a-9830-4201-8b01-47edec592238" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.86:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:13.302213 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:54:13.302178    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:54:13.661956 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:13.661601    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:14.262151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:14.261591    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:14.424665 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:14.424631    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25d54e45-58d2-4476-869c-4b076b617038" path="/var/lib/kubelet/pods/25d54e45-58d2-4476-869c-4b076b617038/volumes"
Oct 30 18:54:15.071610 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:15.071575    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:15.770514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:15.767052    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:16.401120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:16.400215    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:17.213377 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:17.213343    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:17.881146 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:17.881110    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:18.071530 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:18.071494    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:18.283143 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:18.280250    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:19.098488 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:19.094821    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:19.502185 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:19.496541    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:19.685383 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:19.685033    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:19.727317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:19.727274    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"]
Oct 30 18:54:19.727555 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:19.727521    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy" containerID="cri-o://56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55" gracePeriod=30
Oct 30 18:54:19.983116 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:19.982009    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:20.170693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:20.170657    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:20.765121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:20.764150    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:20.869386 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:20.869352    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:20.884532 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:20.884503    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:21.001148 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:21.001116    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:54:21.689700 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:21.689667    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:22.581664 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:22.581627    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:23.269749 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:23.269712    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:23.976930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:23.976895    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:24.587120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:24.584582    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:25.028300 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.028274    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:54:25.030448 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.030418    4925 generic.go:356] "Generic (PLEG): container finished" podID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerID="56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55" exitCode=0
Oct 30 18:54:25.030562 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.030455    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"
Oct 30 18:54:25.030562 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.030464    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" event={"ID":"4e752814-e14b-43f0-9401-71f46afb9a6b","Type":"ContainerDied","Data":"56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55"}
Oct 30 18:54:25.030562 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.030541    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf" event={"ID":"4e752814-e14b-43f0-9401-71f46afb9a6b","Type":"ContainerDied","Data":"f4ddad96139e97835f304b121b1a3b340c06b18156a79b60eedd186de16c2355"}
Oct 30 18:54:25.030719 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.030563    4925 scope.go:122] "RemoveContainer" containerID="56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55"
Oct 30 18:54:25.040485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.040463    4925 scope.go:122] "RemoveContainer" containerID="4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c"
Oct 30 18:54:25.049228 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.049210    4925 scope.go:122] "RemoveContainer" containerID="56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55"
Oct 30 18:54:25.049499 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:54:25.049480    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55\": container with ID starting with 56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55 not found: ID does not exist" containerID="56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55"
Oct 30 18:54:25.049559 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.049508    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55"} err="failed to get container status \"56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55\": rpc error: code = NotFound desc = could not find container \"56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55\": container with ID starting with 56679171dd3f7129331ed0f655f46d49978f968b720a42d8e0c579d00eec3d55 not found: ID does not exist"
Oct 30 18:54:25.049559 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.049524    4925 scope.go:122] "RemoveContainer" containerID="4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c"
Oct 30 18:54:25.049772 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:54:25.049756    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c\": container with ID starting with 4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c not found: ID does not exist" containerID="4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c"
Oct 30 18:54:25.049807 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.049779    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c"} err="failed to get container status \"4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c\": rpc error: code = NotFound desc = could not find container \"4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c\": container with ID starting with 4f3c80b494ee43ea9798f47bd89d57b402df73d4d097f940a667f80bee77db8c not found: ID does not exist"
Oct 30 18:54:25.107472 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.107399    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-socket\") pod \"4e752814-e14b-43f0-9401-71f46afb9a6b\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") "
Oct 30 18:54:25.107672 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.107642    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-socket" pod "4e752814-e14b-43f0-9401-71f46afb9a6b" (UID: "4e752814-e14b-43f0-9401-71f46afb9a6b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:25.107762 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.107662    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-envoy\") pod \"4e752814-e14b-43f0-9401-71f46afb9a6b\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") "
Oct 30 18:54:25.107837 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.107761    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-podinfo\") pod \"4e752814-e14b-43f0-9401-71f46afb9a6b\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") "
Oct 30 18:54:25.107897 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.107859    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-kube-api-access-wc2gl\" (UniqueName: \"kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-kube-api-access-wc2gl\") pod \"4e752814-e14b-43f0-9401-71f46afb9a6b\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") "
Oct 30 18:54:25.107951 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.107928    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-token\" (UniqueName: \"kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-token\") pod \"4e752814-e14b-43f0-9401-71f46afb9a6b\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") "
Oct 30 18:54:25.108342 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.108322    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-data\") pod \"4e752814-e14b-43f0-9401-71f46afb9a6b\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") "
Oct 30 18:54:25.108513 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.108499    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e752814-e14b-43f0-9401-71f46afb9a6b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e752814-e14b-43f0-9401-71f46afb9a6b-istiod-ca-cert\") pod \"4e752814-e14b-43f0-9401-71f46afb9a6b\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") "
Oct 30 18:54:25.108705 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.108691    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-certs\") pod \"4e752814-e14b-43f0-9401-71f46afb9a6b\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") "
Oct 30 18:54:25.108908 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.108894    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-credential-socket\") pod \"4e752814-e14b-43f0-9401-71f46afb9a6b\" (UID: \"4e752814-e14b-43f0-9401-71f46afb9a6b\") "
Oct 30 18:54:25.109235 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.108814    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-data" pod "4e752814-e14b-43f0-9401-71f46afb9a6b" (UID: "4e752814-e14b-43f0-9401-71f46afb9a6b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:25.109317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.109195    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-certs" pod "4e752814-e14b-43f0-9401-71f46afb9a6b" (UID: "4e752814-e14b-43f0-9401-71f46afb9a6b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:25.109317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.109278    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e752814-e14b-43f0-9401-71f46afb9a6b-istiod-ca-cert" pod "4e752814-e14b-43f0-9401-71f46afb9a6b" (UID: "4e752814-e14b-43f0-9401-71f46afb9a6b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:25.109719 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.109694    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-credential-socket" pod "4e752814-e14b-43f0-9401-71f46afb9a6b" (UID: "4e752814-e14b-43f0-9401-71f46afb9a6b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:25.109918 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.109900    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:25.110002 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.109990    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e752814-e14b-43f0-9401-71f46afb9a6b-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:25.110077 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.110067    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:25.110183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.110172    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:25.110256 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.110245    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:25.111007 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.110980    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-envoy" pod "4e752814-e14b-43f0-9401-71f46afb9a6b" (UID: "4e752814-e14b-43f0-9401-71f46afb9a6b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:25.112413 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.111160    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-kube-api-access-wc2gl" pod "4e752814-e14b-43f0-9401-71f46afb9a6b" (UID: "4e752814-e14b-43f0-9401-71f46afb9a6b"). InnerVolumeSpecName "kube-api-access-wc2gl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:25.112547 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.112521    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-podinfo" pod "4e752814-e14b-43f0-9401-71f46afb9a6b" (UID: "4e752814-e14b-43f0-9401-71f46afb9a6b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:54:25.116561 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.116539    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-token" pod "4e752814-e14b-43f0-9401-71f46afb9a6b" (UID: "4e752814-e14b-43f0-9401-71f46afb9a6b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:25.211357 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.211332    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:25.211357 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.211357    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:25.211537 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.211367    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e752814-e14b-43f0-9401-71f46afb9a6b-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:25.211537 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.211378    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc2gl\" (UniqueName: \"kubernetes.io/projected/4e752814-e14b-43f0-9401-71f46afb9a6b-kube-api-access-wc2gl\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:25.342130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.341550    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:25.377429 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.377396    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"]
Oct 30 18:54:25.382397 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.382366    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f37d42-openshift-default-7645c57f7c-hkzkf"]
Oct 30 18:54:25.971119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:25.965746    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:26.423919 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:26.423888    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" path="/var/lib/kubelet/pods/4e752814-e14b-43f0-9401-71f46afb9a6b/volumes"
Oct 30 18:54:26.669211 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:26.667017    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:27.393730 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:27.390676    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:28.204300 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:28.204247    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:28.375743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:28.375711    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:28.823270 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:28.823159    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:29.067141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:29.067084    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:29.429297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:29.429241    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:29.776211 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:29.776045    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:30.370943 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:30.370900    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:30.390880 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:30.390811    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"]
Oct 30 18:54:30.407166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:30.406495    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"]
Oct 30 18:54:30.568140 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:30.568084    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:30.754125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:30.753929    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:31.179903 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:31.162574    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:31.767085 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:31.767050    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:32.400117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:32.392183    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:33.032717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:33.032682    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:33.754464 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:33.754430    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:34.416688 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:34.416644    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:34.958505 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:34.958465    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:35.135725 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.135683    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:35.392182 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.392060    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5" podUID="33d3855a-6e7b-48a4-b842-41d77ea5e317" containerName="payload-processing" containerID="cri-o://738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9" gracePeriod=25
Oct 30 18:54:35.406847 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.406788    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79" podUID="d1090131-35f1-4572-bb89-bd36236c5c66" containerName="payload-pre-processing" containerID="cri-o://dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187" gracePeriod=25
Oct 30 18:54:35.734442 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.734413    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"
Oct 30 18:54:35.809786 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.809750    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33d3855a-6e7b-48a4-b842-41d77ea5e317-kube-api-access-s94l4\" (UniqueName: \"kubernetes.io/projected/33d3855a-6e7b-48a4-b842-41d77ea5e317-kube-api-access-s94l4\") pod \"33d3855a-6e7b-48a4-b842-41d77ea5e317\" (UID: \"33d3855a-6e7b-48a4-b842-41d77ea5e317\") "
Oct 30 18:54:35.810118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.810073    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33d3855a-6e7b-48a4-b842-41d77ea5e317-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d3855a-6e7b-48a4-b842-41d77ea5e317-plugins-config-volume\") pod \"33d3855a-6e7b-48a4-b842-41d77ea5e317\" (UID: \"33d3855a-6e7b-48a4-b842-41d77ea5e317\") "
Oct 30 18:54:35.811077 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.811050    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33d3855a-6e7b-48a4-b842-41d77ea5e317-plugins-config-volume" pod "33d3855a-6e7b-48a4-b842-41d77ea5e317" (UID: "33d3855a-6e7b-48a4-b842-41d77ea5e317"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:35.829005 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.828964    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33d3855a-6e7b-48a4-b842-41d77ea5e317-kube-api-access-s94l4" pod "33d3855a-6e7b-48a4-b842-41d77ea5e317" (UID: "33d3855a-6e7b-48a4-b842-41d77ea5e317"). InnerVolumeSpecName "kube-api-access-s94l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:35.830486 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.830205    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"
Oct 30 18:54:35.920325 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.920290    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1090131-35f1-4572-bb89-bd36236c5c66-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1090131-35f1-4572-bb89-bd36236c5c66-plugins-config-volume\") pod \"d1090131-35f1-4572-bb89-bd36236c5c66\" (UID: \"d1090131-35f1-4572-bb89-bd36236c5c66\") "
Oct 30 18:54:35.920682 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.920649    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1090131-35f1-4572-bb89-bd36236c5c66-kube-api-access-t2rtr\" (UniqueName: \"kubernetes.io/projected/d1090131-35f1-4572-bb89-bd36236c5c66-kube-api-access-t2rtr\") pod \"d1090131-35f1-4572-bb89-bd36236c5c66\" (UID: \"d1090131-35f1-4572-bb89-bd36236c5c66\") "
Oct 30 18:54:35.921560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.921531    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s94l4\" (UniqueName: \"kubernetes.io/projected/33d3855a-6e7b-48a4-b842-41d77ea5e317-kube-api-access-s94l4\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:35.921702 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.921687    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33d3855a-6e7b-48a4-b842-41d77ea5e317-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:35.923796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.923744    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1090131-35f1-4572-bb89-bd36236c5c66-plugins-config-volume" pod "d1090131-35f1-4572-bb89-bd36236c5c66" (UID: "d1090131-35f1-4572-bb89-bd36236c5c66"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:35.928697 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.928500    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:35.969072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:35.968967    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1090131-35f1-4572-bb89-bd36236c5c66-kube-api-access-t2rtr" pod "d1090131-35f1-4572-bb89-bd36236c5c66" (UID: "d1090131-35f1-4572-bb89-bd36236c5c66"). InnerVolumeSpecName "kube-api-access-t2rtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:36.023521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.023487    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2rtr\" (UniqueName: \"kubernetes.io/projected/d1090131-35f1-4572-bb89-bd36236c5c66-kube-api-access-t2rtr\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:36.023521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.023518    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1090131-35f1-4572-bb89-bd36236c5c66-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:36.117775 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.117745    4925 generic.go:356] "Generic (PLEG): container finished" podID="d1090131-35f1-4572-bb89-bd36236c5c66" containerID="dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187" exitCode=0
Oct 30 18:54:36.117982 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.117810    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"
Oct 30 18:54:36.117982 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.117832    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79" event={"ID":"d1090131-35f1-4572-bb89-bd36236c5c66","Type":"ContainerDied","Data":"dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187"}
Oct 30 18:54:36.117982 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.117949    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79" event={"ID":"d1090131-35f1-4572-bb89-bd36236c5c66","Type":"ContainerDied","Data":"b1b81aa947a55cc50e7de35992d881c1754561d78c619e7685715badba2d88d7"}
Oct 30 18:54:36.118185 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.117984    4925 scope.go:122] "RemoveContainer" containerID="dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187"
Oct 30 18:54:36.119419 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.119398    4925 generic.go:356] "Generic (PLEG): container finished" podID="33d3855a-6e7b-48a4-b842-41d77ea5e317" containerID="738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9" exitCode=0
Oct 30 18:54:36.119571 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.119556    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5" event={"ID":"33d3855a-6e7b-48a4-b842-41d77ea5e317","Type":"ContainerDied","Data":"738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9"}
Oct 30 18:54:36.119649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.119599    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5" event={"ID":"33d3855a-6e7b-48a4-b842-41d77ea5e317","Type":"ContainerDied","Data":"9f422539b32c8a17983eb8d6de62fe8fcf566c315d23eacc3e88e52ebcc6acb5"}
Oct 30 18:54:36.119717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.119706    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"
Oct 30 18:54:36.130008 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.129974    4925 scope.go:122] "RemoveContainer" containerID="dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187"
Oct 30 18:54:36.130427 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:54:36.130291    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187\": container with ID starting with dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187 not found: ID does not exist" containerID="dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187"
Oct 30 18:54:36.130427 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.130334    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187"} err="failed to get container status \"dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187\": rpc error: code = NotFound desc = could not find container \"dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187\": container with ID starting with dba9ef2ed40f63ac8f36f4311ac0dbf2a321f662fdd99cafed361be7fff6a187 not found: ID does not exist"
Oct 30 18:54:36.130427 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.130356    4925 scope.go:122] "RemoveContainer" containerID="738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9"
Oct 30 18:54:36.146004 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.145986    4925 scope.go:122] "RemoveContainer" containerID="738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9"
Oct 30 18:54:36.146386 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:54:36.146315    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9\": container with ID starting with 738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9 not found: ID does not exist" containerID="738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9"
Oct 30 18:54:36.146502 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.146350    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9"} err="failed to get container status \"738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9\": rpc error: code = NotFound desc = could not find container \"738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9\": container with ID starting with 738c111be8de98ea23585f9d0cdf8ba6e96d8dc6258f211e7efc892152dd51a9 not found: ID does not exist"
Oct 30 18:54:36.150706 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.150681    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"]
Oct 30 18:54:36.154255 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.154234    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-066928-5866c665f8-pzkx5"]
Oct 30 18:54:36.164710 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.164687    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"]
Oct 30 18:54:36.169330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.169308    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-066928-69f65fb487-fdp79"]
Oct 30 18:54:36.425109 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.425070    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33d3855a-6e7b-48a4-b842-41d77ea5e317" path="/var/lib/kubelet/pods/33d3855a-6e7b-48a4-b842-41d77ea5e317/volumes"
Oct 30 18:54:36.425562 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.425542    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1090131-35f1-4572-bb89-bd36236c5c66" path="/var/lib/kubelet/pods/d1090131-35f1-4572-bb89-bd36236c5c66/volumes"
Oct 30 18:54:36.589461 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:36.589422    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:37.291223 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:37.286637    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:38.068120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:38.066643    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:38.654071 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:38.654030    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:39.266257 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:39.266217    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:39.967152 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:39.966149    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:40.715073 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:40.715001    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:41.424005 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:41.423965    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:42.087186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:42.087148    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:42.470296 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:42.470265    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"]
Oct 30 18:54:42.471116 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:42.471062    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv" podUID="8101b22c-2203-455a-943b-1372e7427e99" containerName="maas-api" containerID="cri-o://c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563" gracePeriod=30
Oct 30 18:54:42.589184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:42.589125    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:42.960721 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:42.960565    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:43.015057 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.015026    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:54:43.043345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.042211    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-maas-api-tls\") pod \"8101b22c-2203-455a-943b-1372e7427e99\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") "
Oct 30 18:54:43.043345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.042494    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-metrics-certs\") pod \"8101b22c-2203-455a-943b-1372e7427e99\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") "
Oct 30 18:54:43.043345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.042531    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8101b22c-2203-455a-943b-1372e7427e99-kube-api-access-q22cn\" (UniqueName: \"kubernetes.io/projected/8101b22c-2203-455a-943b-1372e7427e99-kube-api-access-q22cn\") pod \"8101b22c-2203-455a-943b-1372e7427e99\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") "
Oct 30 18:54:43.043345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.042567    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8101b22c-2203-455a-943b-1372e7427e99-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8101b22c-2203-455a-943b-1372e7427e99-openshift-service-ca\") pod \"8101b22c-2203-455a-943b-1372e7427e99\" (UID: \"8101b22c-2203-455a-943b-1372e7427e99\") "
Oct 30 18:54:43.044461 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.044431    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8101b22c-2203-455a-943b-1372e7427e99-openshift-service-ca" pod "8101b22c-2203-455a-943b-1372e7427e99" (UID: "8101b22c-2203-455a-943b-1372e7427e99"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:43.050158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.050047    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-metrics-certs" pod "8101b22c-2203-455a-943b-1372e7427e99" (UID: "8101b22c-2203-455a-943b-1372e7427e99"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:43.050560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.050534    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-maas-api-tls" pod "8101b22c-2203-455a-943b-1372e7427e99" (UID: "8101b22c-2203-455a-943b-1372e7427e99"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:43.054597 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.054561    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8101b22c-2203-455a-943b-1372e7427e99-kube-api-access-q22cn" pod "8101b22c-2203-455a-943b-1372e7427e99" (UID: "8101b22c-2203-455a-943b-1372e7427e99"). InnerVolumeSpecName "kube-api-access-q22cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:43.144057 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.144022    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:43.144057 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.144057    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q22cn\" (UniqueName: \"kubernetes.io/projected/8101b22c-2203-455a-943b-1372e7427e99-kube-api-access-q22cn\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:43.144301 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.144074    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8101b22c-2203-455a-943b-1372e7427e99-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:43.144301 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.144087    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8101b22c-2203-455a-943b-1372e7427e99-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:43.145293 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.145266    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:43.281176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.279312    4925 generic.go:356] "Generic (PLEG): container finished" podID="8101b22c-2203-455a-943b-1372e7427e99" containerID="c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563" exitCode=0
Oct 30 18:54:43.281176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.279408    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv" event={"ID":"8101b22c-2203-455a-943b-1372e7427e99","Type":"ContainerDied","Data":"c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563"}
Oct 30 18:54:43.281176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.279470    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv" event={"ID":"8101b22c-2203-455a-943b-1372e7427e99","Type":"ContainerDied","Data":"545ffdad01c24c555e1ebb0de9d065ea23a1b004864ebd41dbab496e5f27e586"}
Oct 30 18:54:43.281176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.279505    4925 scope.go:122] "RemoveContainer" containerID="c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563"
Oct 30 18:54:43.281176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.279678    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"
Oct 30 18:54:43.371328 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.371299    4925 scope.go:122] "RemoveContainer" containerID="c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563"
Oct 30 18:54:43.372401 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:54:43.371822    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563\": container with ID starting with c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563 not found: ID does not exist" containerID="c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563"
Oct 30 18:54:43.372401 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.371863    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563"} err="failed to get container status \"c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563\": rpc error: code = NotFound desc = could not find container \"c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563\": container with ID starting with c235a4eabcccb36308c1fdc6e6b49ae030198ac471b75dd8ab465e080c15b563 not found: ID does not exist"
Oct 30 18:54:43.382657 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.381314    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:43.390907 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.390874    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"]
Oct 30 18:54:43.400164 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.400042    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-066928-5549c5cb8b-xrtvv"]
Oct 30 18:54:43.411240 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:54:43.411200    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:54:43.627732 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.627638    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:43.689965 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.689766    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"]
Oct 30 18:54:43.690144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.690021    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9" podUID="f0b83aec-e567-4e29-b17b-3f5c697982e9" containerName="maas-api" containerID="cri-o://f512bf4c4b824dae4dbcb05bf53b1b434963fc983641804376f4705ead414314" gracePeriod=30
Oct 30 18:54:43.808873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:43.808845    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:44.009857 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.009653    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:44.231433 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.230247    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:44.299674 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.299523    4925 generic.go:356] "Generic (PLEG): container finished" podID="f0b83aec-e567-4e29-b17b-3f5c697982e9" containerID="f512bf4c4b824dae4dbcb05bf53b1b434963fc983641804376f4705ead414314" exitCode=0
Oct 30 18:54:44.299905 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.299577    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9" event={"ID":"f0b83aec-e567-4e29-b17b-3f5c697982e9","Type":"ContainerDied","Data":"f512bf4c4b824dae4dbcb05bf53b1b434963fc983641804376f4705ead414314"}
Oct 30 18:54:44.300123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.300078    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9" event={"ID":"f0b83aec-e567-4e29-b17b-3f5c697982e9","Type":"ContainerDied","Data":"ca56c2f1d6b10e1fc09d36a7cf62a2225fd0aee604792212a1f45bfa4bc32b13"}
Oct 30 18:54:44.300278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.300262    4925 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca56c2f1d6b10e1fc09d36a7cf62a2225fd0aee604792212a1f45bfa4bc32b13"
Oct 30 18:54:44.301376 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.301318    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:54:44.538120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.537743    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0b83aec-e567-4e29-b17b-3f5c697982e9-kube-api-access-l77g9\" (UniqueName: \"kubernetes.io/projected/f0b83aec-e567-4e29-b17b-3f5c697982e9-kube-api-access-l77g9\") pod \"f0b83aec-e567-4e29-b17b-3f5c697982e9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") "
Oct 30 18:54:44.538120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.537855    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0b83aec-e567-4e29-b17b-3f5c697982e9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0b83aec-e567-4e29-b17b-3f5c697982e9-openshift-service-ca\") pod \"f0b83aec-e567-4e29-b17b-3f5c697982e9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") "
Oct 30 18:54:44.538120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.537911    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-maas-api-tls\") pod \"f0b83aec-e567-4e29-b17b-3f5c697982e9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") "
Oct 30 18:54:44.538120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.537946    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-metrics-certs\") pod \"f0b83aec-e567-4e29-b17b-3f5c697982e9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") "
Oct 30 18:54:44.616927 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.606860    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:44.702454 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.673993    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-metrics-certs" pod "f0b83aec-e567-4e29-b17b-3f5c697982e9" (UID: "f0b83aec-e567-4e29-b17b-3f5c697982e9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:44.702454 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.674190    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0b83aec-e567-4e29-b17b-3f5c697982e9-kube-api-access-l77g9" pod "f0b83aec-e567-4e29-b17b-3f5c697982e9" (UID: "f0b83aec-e567-4e29-b17b-3f5c697982e9"). InnerVolumeSpecName "kube-api-access-l77g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:44.704605 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.704577    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-maas-api-tls" pod "f0b83aec-e567-4e29-b17b-3f5c697982e9" (UID: "f0b83aec-e567-4e29-b17b-3f5c697982e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:44.780025 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.779992    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0b83aec-e567-4e29-b17b-3f5c697982e9-openshift-service-ca" pod "f0b83aec-e567-4e29-b17b-3f5c697982e9" (UID: "f0b83aec-e567-4e29-b17b-3f5c697982e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:44.794359 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.794321    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0b83aec-e567-4e29-b17b-3f5c697982e9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0b83aec-e567-4e29-b17b-3f5c697982e9-openshift-service-ca\") pod \"f0b83aec-e567-4e29-b17b-3f5c697982e9\" (UID: \"f0b83aec-e567-4e29-b17b-3f5c697982e9\") "
Oct 30 18:54:44.795383 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.795354    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l77g9\" (UniqueName: \"kubernetes.io/projected/f0b83aec-e567-4e29-b17b-3f5c697982e9-kube-api-access-l77g9\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:44.795494 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.795391    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:44.795494 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.795410    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0b83aec-e567-4e29-b17b-3f5c697982e9-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:44.795629 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:54:44.795504    4925 empty_dir.go:505] Warning: Unmount skipped because path does not exist: /var/lib/kubelet/pods/f0b83aec-e567-4e29-b17b-3f5c697982e9/volumes/kubernetes.io~configmap/openshift-service-ca
Oct 30 18:54:44.795629 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.795518    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0b83aec-e567-4e29-b17b-3f5c697982e9-openshift-service-ca" pod "f0b83aec-e567-4e29-b17b-3f5c697982e9" (UID: "f0b83aec-e567-4e29-b17b-3f5c697982e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:44.912121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.908579    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0b83aec-e567-4e29-b17b-3f5c697982e9-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:44.938200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.935384    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:44.938200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:44.937310    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8101b22c-2203-455a-943b-1372e7427e99" path="/var/lib/kubelet/pods/8101b22c-2203-455a-943b-1372e7427e99/volumes"
Oct 30 18:54:45.341172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:45.339015    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"
Oct 30 18:54:45.377886 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:45.377857    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"]
Oct 30 18:54:45.396137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:45.396109    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-7784e5-6b4bc7cc59-kspk9"]
Oct 30 18:54:46.072121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:46.070629    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:46.424885 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:46.424846    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0b83aec-e567-4e29-b17b-3f5c697982e9" path="/var/lib/kubelet/pods/f0b83aec-e567-4e29-b17b-3f5c697982e9/volumes"
Oct 30 18:54:46.816202 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:46.807748    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:47.576119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:47.570420    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:48.181602 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:48.181571    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:48.814756 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:48.814715    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:49.576329 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:49.576296    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:50.009834 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:49.999001    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:50.182409 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:50.182379    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:50.385896 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:50.385809    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:51.093167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:51.090396    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:51.678690 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:51.678654    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:51.873355 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:51.873320    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:52.063272 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:52.060513    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:52.750204 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:52.750169    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:53.399220 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:53.399179    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:53.907225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:53.907190    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:54.094292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.093248    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.217292    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"]
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.217964    4925 cpu_manager.go:414] "containerMap: removing container" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.217978    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.217989    4925 cpu_manager.go:414] "containerMap: removing container" podUID="f0b83aec-e567-4e29-b17b-3f5c697982e9" containerName="maas-api"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.217998    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0b83aec-e567-4e29-b17b-3f5c697982e9" containerName="maas-api"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218013    4925 cpu_manager.go:414] "containerMap: removing container" podUID="33d3855a-6e7b-48a4-b842-41d77ea5e317" containerName="payload-processing"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218018    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33d3855a-6e7b-48a4-b842-41d77ea5e317" containerName="payload-processing"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218025    4925 cpu_manager.go:414] "containerMap: removing container" podUID="8101b22c-2203-455a-943b-1372e7427e99" containerName="maas-api"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218030    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8101b22c-2203-455a-943b-1372e7427e99" containerName="maas-api"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218040    4925 cpu_manager.go:414] "containerMap: removing container" podUID="25d54e45-58d2-4476-869c-4b076b617038" containerName="maas-api"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218045    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25d54e45-58d2-4476-869c-4b076b617038" containerName="maas-api"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218050    4925 cpu_manager.go:414] "containerMap: removing container" podUID="d1090131-35f1-4572-bb89-bd36236c5c66" containerName="payload-pre-processing"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218055    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1090131-35f1-4572-bb89-bd36236c5c66" containerName="payload-pre-processing"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218121    4925 cpu_manager.go:414] "containerMap: removing container" podUID="ffa5c77a-9830-4201-8b01-47edec592238" containerName="maas-api"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218127    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffa5c77a-9830-4201-8b01-47edec592238" containerName="maas-api"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218226    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffa5c77a-9830-4201-8b01-47edec592238" containerName="maas-api"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218242    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218249    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0b83aec-e567-4e29-b17b-3f5c697982e9" containerName="maas-api"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218256    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="33d3855a-6e7b-48a4-b842-41d77ea5e317" containerName="payload-processing"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218261    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="8101b22c-2203-455a-943b-1372e7427e99" containerName="maas-api"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218268    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="25d54e45-58d2-4476-869c-4b076b617038" containerName="maas-api"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218274    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1090131-35f1-4572-bb89-bd36236c5c66" containerName="payload-pre-processing"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.218284    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy"
Oct 30 18:54:54.223757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.223464    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.235239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.233250    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-dc0f87-openshift-default-dockercfg-98s2z\""
Oct 30 18:54:54.235239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.234603    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"]
Oct 30 18:54:54.355295 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.354028    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:54.367124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.363306    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-envoy\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.367524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.367492    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-token\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.367685 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.367612    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2jvf\" (UniqueName: \"kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-kube-api-access-h2jvf\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.367754 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.367702    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-credential-socket\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.367813 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.367772    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-certs\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.367869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.367839    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-podinfo\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.367921 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.367869    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-socket\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.367921 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.367900    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086f5d84-6529-4e59-bbc9-ec0141cd16de-istiod-ca-cert\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.368085 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.367948    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-data\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.497169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.489454    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-envoy\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.497169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.489532    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-token\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.497169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.489664    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2jvf\" (UniqueName: \"kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-kube-api-access-h2jvf\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.497169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.489757    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-credential-socket\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.497169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.489946    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-certs\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.497169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.490231    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-podinfo\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.497169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.490272    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-socket\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.497169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.490306    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086f5d84-6529-4e59-bbc9-ec0141cd16de-istiod-ca-cert\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.497169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.490388    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-data\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.497169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.490912    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-data\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.497169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.491957    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-certs\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.508164 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.500170    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-credential-socket\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.508164 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.500642    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-socket\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.509014 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.508988    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-envoy\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.509489 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.509464    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-podinfo\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.516205 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.509867    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086f5d84-6529-4e59-bbc9-ec0141cd16de-istiod-ca-cert\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.564503 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.564458    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2jvf\" (UniqueName: \"kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-kube-api-access-h2jvf\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.568275 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.568248    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-token\") pod \"e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") " pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:54.812198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.806645    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:54.842365 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:54.842327    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:55.087617 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:55.084463    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:55.137188 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:55.137159    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"]
Oct 30 18:54:55.138428 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:54:55.138385    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086f5d84_6529_4e59_bbc9_ec0141cd16de.slice/crio-4a456917d94633ac29452e0f1859463f466aa9dfe15ed77b3cc0b54d8ee8e80e WatchSource:0}: Error finding container 4a456917d94633ac29452e0f1859463f466aa9dfe15ed77b3cc0b54d8ee8e80e: Status 404 returned error can't find the container with id 4a456917d94633ac29452e0f1859463f466aa9dfe15ed77b3cc0b54d8ee8e80e
Oct 30 18:54:55.141340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:55.141306    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:54:55.141414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:55.141394    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:54:55.141448 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:55.141434    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:54:55.584420 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:55.584370    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" event={"ID":"086f5d84-6529-4e59-bbc9-ec0141cd16de","Type":"ContainerStarted","Data":"852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442"}
Oct 30 18:54:55.584640 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:55.584460    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" event={"ID":"086f5d84-6529-4e59-bbc9-ec0141cd16de","Type":"ContainerStarted","Data":"4a456917d94633ac29452e0f1859463f466aa9dfe15ed77b3cc0b54d8ee8e80e"}
Oct 30 18:54:55.607213 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:55.607168    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" podStartSLOduration=1.607156332 podStartE2EDuration="1.607156332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:55.60481566 +0000 UTC m=+1117.321160739" watchObservedRunningTime="2026-10-30 18:54:55.607156332 +0000 UTC m=+1117.323501408"
Oct 30 18:54:55.689627 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:55.689594    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:55.842842 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:55.842755    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:56.394124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:56.387243    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:56.848201 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:56.848170    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:57.069013 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:57.068978    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:57.597452 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:57.597417    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:57.598469 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:57.598445    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:54:57.769087 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:57.769051    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:57.947499 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:57.947449    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:58.141731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:58.141698    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:58.642501 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:58.642410    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:59.220797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.220771    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:59.235186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.235155    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"]
Oct 30 18:54:59.235828 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.235812    4925 cpu_manager.go:414] "containerMap: removing container" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy"
Oct 30 18:54:59.235877 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.235830    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e752814-e14b-43f0-9401-71f46afb9a6b" containerName="istio-proxy"
Oct 30 18:54:59.239629 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.239609    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"
Oct 30 18:54:59.246996 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.246970    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-dc0f87\""
Oct 30 18:54:59.246996 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.246982    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-dc0f87-dockercfg-8mc59\""
Oct 30 18:54:59.253539 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.251615    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"]
Oct 30 18:54:59.256080 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.256063    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"]
Oct 30 18:54:59.261229 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.261204    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"
Oct 30 18:54:59.275520 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.275501    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"]
Oct 30 18:54:59.288320 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.287499    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-plugins-config-volume\") pod \"payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c\" (UID: \"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982\") " pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"
Oct 30 18:54:59.288320 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.287695    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d258f72d-529b-46e6-9637-4f947f5c12b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t\" (UID: \"d258f72d-529b-46e6-9637-4f947f5c12b4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"
Oct 30 18:54:59.288320 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.287761    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llrnc\" (UniqueName: \"kubernetes.io/projected/d258f72d-529b-46e6-9637-4f947f5c12b4-kube-api-access-llrnc\") pod \"payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t\" (UID: \"d258f72d-529b-46e6-9637-4f947f5c12b4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"
Oct 30 18:54:59.288320 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.287831    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kb6x\" (UniqueName: \"kubernetes.io/projected/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-kube-api-access-6kb6x\") pod \"payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c\" (UID: \"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982\") " pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"
Oct 30 18:54:59.403120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.402995    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-plugins-config-volume\") pod \"payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c\" (UID: \"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982\") " pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"
Oct 30 18:54:59.403323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.403231    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d258f72d-529b-46e6-9637-4f947f5c12b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t\" (UID: \"d258f72d-529b-46e6-9637-4f947f5c12b4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"
Oct 30 18:54:59.403323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.403291    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llrnc\" (UniqueName: \"kubernetes.io/projected/d258f72d-529b-46e6-9637-4f947f5c12b4-kube-api-access-llrnc\") pod \"payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t\" (UID: \"d258f72d-529b-46e6-9637-4f947f5c12b4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"
Oct 30 18:54:59.403434 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.403324    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kb6x\" (UniqueName: \"kubernetes.io/projected/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-kube-api-access-6kb6x\") pod \"payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c\" (UID: \"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982\") " pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"
Oct 30 18:54:59.426118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.413648    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-plugins-config-volume\") pod \"payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c\" (UID: \"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982\") " pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"
Oct 30 18:54:59.426118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.414424    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d258f72d-529b-46e6-9637-4f947f5c12b4-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t\" (UID: \"d258f72d-529b-46e6-9637-4f947f5c12b4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"
Oct 30 18:54:59.444134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.442778    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kb6x\" (UniqueName: \"kubernetes.io/projected/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-kube-api-access-6kb6x\") pod \"payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c\" (UID: \"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982\") " pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"
Oct 30 18:54:59.471255 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.467304    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:59.479118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.476798    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llrnc\" (UniqueName: \"kubernetes.io/projected/d258f72d-529b-46e6-9637-4f947f5c12b4-kube-api-access-llrnc\") pod \"payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t\" (UID: \"d258f72d-529b-46e6-9637-4f947f5c12b4\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"
Oct 30 18:54:59.564122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.563598    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"
Oct 30 18:54:59.600992 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.599343    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"
Oct 30 18:54:59.898356 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.898313    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"]
Oct 30 18:54:59.901759 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:54:59.901718    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd258f72d_529b_46e6_9637_4f947f5c12b4.slice/crio-f26198853bc8864a6f2806a32c8b48e25e50db7cc30ba7eaa59ec93efa867d34 WatchSource:0}: Error finding container f26198853bc8864a6f2806a32c8b48e25e50db7cc30ba7eaa59ec93efa867d34: Status 404 returned error can't find the container with id f26198853bc8864a6f2806a32c8b48e25e50db7cc30ba7eaa59ec93efa867d34
Oct 30 18:54:59.915251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:54:59.915227    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"]
Oct 30 18:54:59.918464 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:54:59.918438    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbbc3b1e_bb21_4ca4_9007_d5a0d25e4982.slice/crio-07c627cdc8ac31b22a6fc3f4fc91b846bdd9678cb028a79855e31f0599ab6ce0 WatchSource:0}: Error finding container 07c627cdc8ac31b22a6fc3f4fc91b846bdd9678cb028a79855e31f0599ab6ce0: Status 404 returned error can't find the container with id 07c627cdc8ac31b22a6fc3f4fc91b846bdd9678cb028a79855e31f0599ab6ce0
Oct 30 18:55:00.183144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:00.178732    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:00.634776 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:00.634737    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t" event={"ID":"d258f72d-529b-46e6-9637-4f947f5c12b4","Type":"ContainerStarted","Data":"6c8e2c6a0925cabce79ea4c6ca2fd649f84f892226b6278c0cfaa68024af8b3d"}
Oct 30 18:55:00.634955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:00.634814    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"
Oct 30 18:55:00.634955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:00.634840    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t" event={"ID":"d258f72d-529b-46e6-9637-4f947f5c12b4","Type":"ContainerStarted","Data":"f26198853bc8864a6f2806a32c8b48e25e50db7cc30ba7eaa59ec93efa867d34"}
Oct 30 18:55:00.636339 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:00.636314    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c" event={"ID":"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982","Type":"ContainerStarted","Data":"7e4faa0adb98884aa245b2716938e26f12638420dc5881110049bf9638495b2f"}
Oct 30 18:55:00.636452 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:00.636361    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c" event={"ID":"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982","Type":"ContainerStarted","Data":"07c627cdc8ac31b22a6fc3f4fc91b846bdd9678cb028a79855e31f0599ab6ce0"}
Oct 30 18:55:00.636452 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:00.636397    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"
Oct 30 18:55:00.653851 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:00.653813    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t" podStartSLOduration=1.653800996 podStartE2EDuration="1.653800996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:00.651015497 +0000 UTC m=+1122.367360573" watchObservedRunningTime="2026-10-30 18:55:00.653800996 +0000 UTC m=+1122.370146066"
Oct 30 18:55:00.667329 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:00.667277    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c" podStartSLOduration=1.667259643 podStartE2EDuration="1.667259643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:00.666294198 +0000 UTC m=+1122.382639286" watchObservedRunningTime="2026-10-30 18:55:00.667259643 +0000 UTC m=+1122.383604720"
Oct 30 18:55:00.880364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:00.880321    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:01.555135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:01.554961    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:02.114664 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.114617    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"]
Oct 30 18:55:02.120420 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.120391    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.128404 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.128379    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-dc0f87-serving-cert\""
Oct 30 18:55:02.133882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.133859    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"]
Oct 30 18:55:02.169011 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.168971    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9rwt\" (UniqueName: \"kubernetes.io/projected/9379191b-9d56-408d-9b33-215e889c7713-kube-api-access-b9rwt\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.169182 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.169076    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-metrics-certs\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.169260 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.169244    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-maas-api-tls\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.169312 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.169298    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9379191b-9d56-408d-9b33-215e889c7713-openshift-service-ca\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.272145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.271665    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:02.276939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.273427    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9rwt\" (UniqueName: \"kubernetes.io/projected/9379191b-9d56-408d-9b33-215e889c7713-kube-api-access-b9rwt\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.276939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.273576    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-metrics-certs\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.276939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.273743    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-maas-api-tls\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.276939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.273800    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9379191b-9d56-408d-9b33-215e889c7713-openshift-service-ca\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.287377 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:55:02.285474    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-serving-cert: secret "maas-api-e2e-api-a-dc0f87-serving-cert" not found
Oct 30 18:55:02.287377 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:55:02.285555    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-maas-api-tls podName:9379191b-9d56-408d-9b33-215e889c7713 nodeName:}" failed. No retries permitted until 2026-10-30 18:55:02.785532948 +0000 UTC m=+1124.501878003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-maas-api-tls") pod "maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8" (UID: "9379191b-9d56-408d-9b33-215e889c7713") : secret "maas-api-e2e-api-a-dc0f87-serving-cert" not found
Oct 30 18:55:02.309843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.297388    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9379191b-9d56-408d-9b33-215e889c7713-openshift-service-ca\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.315929 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.315897    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9rwt\" (UniqueName: \"kubernetes.io/projected/9379191b-9d56-408d-9b33-215e889c7713-kube-api-access-b9rwt\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.347824 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.347765    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-metrics-certs\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.451034 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.450998    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:02.629056 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.629007    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:02.886136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.886031    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-maas-api-tls\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:02.888965 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:02.888929    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-maas-api-tls\") pod \"maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:03.012119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:03.008803    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:03.051122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:03.048373    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:03.375921 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:03.375879    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"]
Oct 30 18:55:03.377370 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:55:03.377337    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9379191b_9d56_408d_9b33_215e889c7713.slice/crio-111efd2a68e161961f8a90a8d36ac52eee2f5967d85ca212b8db24bf5ad4a3b5 WatchSource:0}: Error finding container 111efd2a68e161961f8a90a8d36ac52eee2f5967d85ca212b8db24bf5ad4a3b5: Status 404 returned error can't find the container with id 111efd2a68e161961f8a90a8d36ac52eee2f5967d85ca212b8db24bf5ad4a3b5
Oct 30 18:55:03.661538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:03.661498    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8" event={"ID":"9379191b-9d56-408d-9b33-215e889c7713","Type":"ContainerStarted","Data":"111efd2a68e161961f8a90a8d36ac52eee2f5967d85ca212b8db24bf5ad4a3b5"}
Oct 30 18:55:03.851123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:03.844557    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:04.486124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:04.484629    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:04.671687 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:04.671653    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8" event={"ID":"9379191b-9d56-408d-9b33-215e889c7713","Type":"ContainerStarted","Data":"d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810"}
Oct 30 18:55:04.672072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:04.671737    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:04.692912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:04.692862    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8" podStartSLOduration=1.972939887 podStartE2EDuration="2.692849745s" totalImagesPullingTime="719.909858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:04.690519326 +0000 UTC m=+1126.406864403" watchObservedRunningTime="2026-10-30 18:55:04.692849745 +0000 UTC m=+1126.409194820"
Oct 30 18:55:05.322195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:05.320003    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:06.217842 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:06.217812    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:06.961168 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:06.961133    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:07.759122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:07.756151    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:08.483808 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:08.483773    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:08.787727 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:08.787626    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:09.150009 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:09.145482    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:09.878123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:09.875050    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:10.488733 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:10.488628    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:10.686192 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:10.686160    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:55:11.129981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:11.111598    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:11.674076 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:11.674043    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"
Oct 30 18:55:11.678725 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:11.678699    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"
Oct 30 18:55:11.689644 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:11.689562    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:12.404122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:12.398798    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:13.121453 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:13.120344    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:13.304523 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:55:13.304485    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:55:13.781941 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:13.781894    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:14.736122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:14.733213    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:15.366654 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:15.365933    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:15.977299 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:15.977249    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:16.570300 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:16.570267    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:17.170045 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:17.169992    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:17.906751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:17.906718    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:18.091951 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:18.091912    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:18.588584 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:18.588550    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:18.771769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:18.771734    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:19.115664 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:19.115541    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:19.511090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:19.511058    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:19.689521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:19.689485    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:20.182638 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:20.182607    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:20.906138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:20.906082    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:20.923146 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:20.923063    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7496-11401a"]
Oct 30 18:55:20.950287 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:20.950257    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7496-11401a"]
Oct 30 18:55:20.950532 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:20.950402    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-11401a"
Oct 30 18:55:20.957756 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:20.957734    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-c64f8\""
Oct 30 18:55:21.023728 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:21.023700    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6smd\" (UniqueName: \"kubernetes.io/projected/7fd531ef-8c6d-47dd-a840-ecc00a376373-kube-api-access-m6smd\") pod \"test-curl-7496-11401a\" (UID: \"7fd531ef-8c6d-47dd-a840-ecc00a376373\") " pod="opendatahub/test-curl-7496-11401a"
Oct 30 18:55:21.125682 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:21.125649    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6smd\" (UniqueName: \"kubernetes.io/projected/7fd531ef-8c6d-47dd-a840-ecc00a376373-kube-api-access-m6smd\") pod \"test-curl-7496-11401a\" (UID: \"7fd531ef-8c6d-47dd-a840-ecc00a376373\") " pod="opendatahub/test-curl-7496-11401a"
Oct 30 18:55:21.133777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:21.133747    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6smd\" (UniqueName: \"kubernetes.io/projected/7fd531ef-8c6d-47dd-a840-ecc00a376373-kube-api-access-m6smd\") pod \"test-curl-7496-11401a\" (UID: \"7fd531ef-8c6d-47dd-a840-ecc00a376373\") " pod="opendatahub/test-curl-7496-11401a"
Oct 30 18:55:21.272105 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:21.272075    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-11401a"
Oct 30 18:55:21.568154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:21.568119    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7496-11401a"]
Oct 30 18:55:21.569407 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:55:21.569364    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fd531ef_8c6d_47dd_a840_ecc00a376373.slice/crio-a516a47417852e3a7bef75b27814e44c7851e233b09c6c6109052331f11fb649 WatchSource:0}: Error finding container a516a47417852e3a7bef75b27814e44c7851e233b09c6c6109052331f11fb649: Status 404 returned error can't find the container with id a516a47417852e3a7bef75b27814e44c7851e233b09c6c6109052331f11fb649
Oct 30 18:55:21.832166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:21.828649    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:21.908049 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:21.908014    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7496-11401a" event={"ID":"7fd531ef-8c6d-47dd-a840-ecc00a376373","Type":"ContainerStarted","Data":"a516a47417852e3a7bef75b27814e44c7851e233b09c6c6109052331f11fb649"}
Oct 30 18:55:22.597122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:22.591725    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:22.789026 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:22.788957    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:23.467691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:23.466826    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:23.840202 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:23.840123    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"]
Oct 30 18:55:23.849561 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:23.849531    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:23.851797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:23.851779    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"]
Oct 30 18:55:23.852215 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:23.851956    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"
Oct 30 18:55:23.864141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:23.861519    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-9982ad-dockercfg-rhzdf\""
Oct 30 18:55:23.864141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:23.862136    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-9982ad\""
Oct 30 18:55:23.873859 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:23.873828    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"]
Oct 30 18:55:23.882838 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:23.882806    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"
Oct 30 18:55:23.904550 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:23.904522    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp\" (UID: \"5bfe6c2a-58d2-4a50-bedf-ef345ada9266\") " pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"
Oct 30 18:55:23.906564 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:23.905060    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpqhw\" (UniqueName: \"kubernetes.io/projected/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-kube-api-access-bpqhw\") pod \"payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp\" (UID: \"5bfe6c2a-58d2-4a50-bedf-ef345ada9266\") " pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"
Oct 30 18:55:23.911848 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:23.911792    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"]
Oct 30 18:55:24.015123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.014264    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrxwm\" (UniqueName: \"kubernetes.io/projected/af7e9f27-86db-4f00-a966-412d1c25eb2f-kube-api-access-mrxwm\") pod \"payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6\" (UID: \"af7e9f27-86db-4f00-a966-412d1c25eb2f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"
Oct 30 18:55:24.015123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.014370    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpqhw\" (UniqueName: \"kubernetes.io/projected/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-kube-api-access-bpqhw\") pod \"payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp\" (UID: \"5bfe6c2a-58d2-4a50-bedf-ef345ada9266\") " pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"
Oct 30 18:55:24.015123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.014515    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af7e9f27-86db-4f00-a966-412d1c25eb2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6\" (UID: \"af7e9f27-86db-4f00-a966-412d1c25eb2f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"
Oct 30 18:55:24.015123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.014668    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp\" (UID: \"5bfe6c2a-58d2-4a50-bedf-ef345ada9266\") " pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"
Oct 30 18:55:24.083134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.073268    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:24.083134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.081165    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-plugins-config-volume\") pod \"payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp\" (UID: \"5bfe6c2a-58d2-4a50-bedf-ef345ada9266\") " pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"
Oct 30 18:55:24.158245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.130109    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af7e9f27-86db-4f00-a966-412d1c25eb2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6\" (UID: \"af7e9f27-86db-4f00-a966-412d1c25eb2f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"
Oct 30 18:55:24.158245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.130331    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrxwm\" (UniqueName: \"kubernetes.io/projected/af7e9f27-86db-4f00-a966-412d1c25eb2f-kube-api-access-mrxwm\") pod \"payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6\" (UID: \"af7e9f27-86db-4f00-a966-412d1c25eb2f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"
Oct 30 18:55:24.158245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.131830    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af7e9f27-86db-4f00-a966-412d1c25eb2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6\" (UID: \"af7e9f27-86db-4f00-a966-412d1c25eb2f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"
Oct 30 18:55:24.177134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.163401    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpqhw\" (UniqueName: \"kubernetes.io/projected/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-kube-api-access-bpqhw\") pod \"payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp\" (UID: \"5bfe6c2a-58d2-4a50-bedf-ef345ada9266\") " pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"
Oct 30 18:55:24.209382 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.201890    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrxwm\" (UniqueName: \"kubernetes.io/projected/af7e9f27-86db-4f00-a966-412d1c25eb2f-kube-api-access-mrxwm\") pod \"payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6\" (UID: \"af7e9f27-86db-4f00-a966-412d1c25eb2f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"
Oct 30 18:55:24.215609 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.215361    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"
Oct 30 18:55:24.216057 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.215962    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"
Oct 30 18:55:24.537472 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.537308    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:24.559340 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:55:24.559312    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf7e9f27_86db_4f00_a966_412d1c25eb2f.slice/crio-c0b646e63104893023ce593e51a619365b0e38d396c7dafd0104e7af09fe5fbb WatchSource:0}: Error finding container c0b646e63104893023ce593e51a619365b0e38d396c7dafd0104e7af09fe5fbb: Status 404 returned error can't find the container with id c0b646e63104893023ce593e51a619365b0e38d396c7dafd0104e7af09fe5fbb
Oct 30 18:55:24.564602 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.564555    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"]
Oct 30 18:55:24.599300 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:24.599270    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"]
Oct 30 18:55:25.270016 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:25.269977    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:25.310118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:25.309686    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6" event={"ID":"af7e9f27-86db-4f00-a966-412d1c25eb2f","Type":"ContainerStarted","Data":"7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220"}
Oct 30 18:55:25.310118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:25.309788    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6" event={"ID":"af7e9f27-86db-4f00-a966-412d1c25eb2f","Type":"ContainerStarted","Data":"c0b646e63104893023ce593e51a619365b0e38d396c7dafd0104e7af09fe5fbb"}
Oct 30 18:55:25.311361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:25.311333    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"
Oct 30 18:55:25.315656 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:25.315623    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7496-11401a" event={"ID":"7fd531ef-8c6d-47dd-a840-ecc00a376373","Type":"ContainerStarted","Data":"15ec0916fbe98ed135dbb6d88c278d14837f1af511a86f580caf1df178e8bac9"}
Oct 30 18:55:25.320147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:25.319038    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp" event={"ID":"5bfe6c2a-58d2-4a50-bedf-ef345ada9266","Type":"ContainerStarted","Data":"8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd"}
Oct 30 18:55:25.320147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:25.319088    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp" event={"ID":"5bfe6c2a-58d2-4a50-bedf-ef345ada9266","Type":"ContainerStarted","Data":"99a2efc62a0f84b69db43f0574f373455453362692d665e13c9eecf3320cc55d"}
Oct 30 18:55:25.320147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:25.319231    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"
Oct 30 18:55:25.332817 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:25.332664    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6" podStartSLOduration=2.33265189 podStartE2EDuration="2.33265189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:25.327478745 +0000 UTC m=+1147.043823836" watchObservedRunningTime="2026-10-30 18:55:25.33265189 +0000 UTC m=+1147.048997002"
Oct 30 18:55:25.344900 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:25.344851    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp" podStartSLOduration=2.344837853 podStartE2EDuration="2.344837853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:25.342990378 +0000 UTC m=+1147.059335457" watchObservedRunningTime="2026-10-30 18:55:25.344837853 +0000 UTC m=+1147.061182929"
Oct 30 18:55:25.361180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:25.361134    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7496-11401a" podStartSLOduration=2.094946673 podStartE2EDuration="5.361122764s" totalImagesPullingTime="3.266176091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:25.357720969 +0000 UTC m=+1147.074066046" watchObservedRunningTime="2026-10-30 18:55:25.361122764 +0000 UTC m=+1147.077467837"
Oct 30 18:55:25.942121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:25.939453    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:26.037652 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:26.034018    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7496-11401a"]
Oct 30 18:55:26.037652 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:26.037558    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7496-11401a"]
Oct 30 18:55:27.068988 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:27.068912    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:27.333032 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:27.332935    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7496-11401a" podUID="7fd531ef-8c6d-47dd-a840-ecc00a376373" containerName="test-curl-7496-11401a" containerID="cri-o://15ec0916fbe98ed135dbb6d88c278d14837f1af511a86f580caf1df178e8bac9" gracePeriod=2
Oct 30 18:55:27.830118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:27.829680    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:28.335031 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.334994    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"]
Oct 30 18:55:28.336076 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.336052    4925 cpu_manager.go:414] "containerMap: removing container" podUID="7fd531ef-8c6d-47dd-a840-ecc00a376373" containerName="test-curl-7496-11401a"
Oct 30 18:55:28.336076 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.336075    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7fd531ef-8c6d-47dd-a840-ecc00a376373" containerName="test-curl-7496-11401a"
Oct 30 18:55:28.336731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.336706    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="7fd531ef-8c6d-47dd-a840-ecc00a376373" containerName="test-curl-7496-11401a"
Oct 30 18:55:28.341867 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.341845    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.357565 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.356450    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-9982ad-serving-cert\""
Oct 30 18:55:28.363331 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.363295    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"]
Oct 30 18:55:28.380547 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.380509    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x4rh\" (UniqueName: \"kubernetes.io/projected/c938f52d-8915-41a8-968d-253e99f1f490-kube-api-access-8x4rh\") pod \"maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.380821 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.380785    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c938f52d-8915-41a8-968d-253e99f1f490-openshift-service-ca\") pod \"maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.380888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.380859    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-maas-api-tls\") pod \"maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.380931 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.380902    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-metrics-certs\") pod \"maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.464994 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.464962    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:28.483031 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.482994    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c938f52d-8915-41a8-968d-253e99f1f490-openshift-service-ca\") pod \"maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.483218 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.483074    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-maas-api-tls\") pod \"maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.483218 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.483131    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-metrics-certs\") pod \"maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.483338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.483287    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x4rh\" (UniqueName: \"kubernetes.io/projected/c938f52d-8915-41a8-968d-253e99f1f490-kube-api-access-8x4rh\") pod \"maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.483666 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.483640    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c938f52d-8915-41a8-968d-253e99f1f490-openshift-service-ca\") pod \"maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.485965 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.485941    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-metrics-certs\") pod \"maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.486076 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.485985    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-maas-api-tls\") pod \"maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.491382 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.491357    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x4rh\" (UniqueName: \"kubernetes.io/projected/c938f52d-8915-41a8-968d-253e99f1f490-kube-api-access-8x4rh\") pod \"maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:28.652174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.651760    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:28.687118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:28.686469    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:29.216172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:29.210202    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"]
Oct 30 18:55:29.260036 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:55:29.252303    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc938f52d_8915_41a8_968d_253e99f1f490.slice/crio-7981be0a735397bd8ad447d8a3a140c27402f5ab5a88037c08867c945eb45a3d WatchSource:0}: Error finding container 7981be0a735397bd8ad447d8a3a140c27402f5ab5a88037c08867c945eb45a3d: Status 404 returned error can't find the container with id 7981be0a735397bd8ad447d8a3a140c27402f5ab5a88037c08867c945eb45a3d
Oct 30 18:55:29.370964 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:29.370924    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw" event={"ID":"c938f52d-8915-41a8-968d-253e99f1f490","Type":"ContainerStarted","Data":"7981be0a735397bd8ad447d8a3a140c27402f5ab5a88037c08867c945eb45a3d"}
Oct 30 18:55:29.616830 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:29.616767    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-11401a"
Oct 30 18:55:29.657125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:29.654204    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:29.716854 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:29.714005    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7fd531ef-8c6d-47dd-a840-ecc00a376373-kube-api-access-m6smd\" (UniqueName: \"kubernetes.io/projected/7fd531ef-8c6d-47dd-a840-ecc00a376373-kube-api-access-m6smd\") pod \"7fd531ef-8c6d-47dd-a840-ecc00a376373\" (UID: \"7fd531ef-8c6d-47dd-a840-ecc00a376373\") "
Oct 30 18:55:29.717462 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:29.717420    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fd531ef-8c6d-47dd-a840-ecc00a376373-kube-api-access-m6smd" pod "7fd531ef-8c6d-47dd-a840-ecc00a376373" (UID: "7fd531ef-8c6d-47dd-a840-ecc00a376373"). InnerVolumeSpecName "kube-api-access-m6smd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:55:29.815873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:29.815844    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6smd\" (UniqueName: \"kubernetes.io/projected/7fd531ef-8c6d-47dd-a840-ecc00a376373-kube-api-access-m6smd\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:55:30.401746 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:30.401704    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:30.407807 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:30.407759    4925 generic.go:356] "Generic (PLEG): container finished" podID="7fd531ef-8c6d-47dd-a840-ecc00a376373" containerID="15ec0916fbe98ed135dbb6d88c278d14837f1af511a86f580caf1df178e8bac9" exitCode=137
Oct 30 18:55:30.408024 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:30.407885    4925 scope.go:122] "RemoveContainer" containerID="15ec0916fbe98ed135dbb6d88c278d14837f1af511a86f580caf1df178e8bac9"
Oct 30 18:55:30.408126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:30.408080    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-11401a"
Oct 30 18:55:30.421916 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:30.421892    4925 scope.go:122] "RemoveContainer" containerID="15ec0916fbe98ed135dbb6d88c278d14837f1af511a86f580caf1df178e8bac9"
Oct 30 18:55:30.423690 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:55:30.423615    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15ec0916fbe98ed135dbb6d88c278d14837f1af511a86f580caf1df178e8bac9\": container with ID starting with 15ec0916fbe98ed135dbb6d88c278d14837f1af511a86f580caf1df178e8bac9 not found: ID does not exist" containerID="15ec0916fbe98ed135dbb6d88c278d14837f1af511a86f580caf1df178e8bac9"
Oct 30 18:55:30.423690 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:30.423658    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15ec0916fbe98ed135dbb6d88c278d14837f1af511a86f580caf1df178e8bac9"} err="failed to get container status \"15ec0916fbe98ed135dbb6d88c278d14837f1af511a86f580caf1df178e8bac9\": rpc error: code = NotFound desc = could not find container \"15ec0916fbe98ed135dbb6d88c278d14837f1af511a86f580caf1df178e8bac9\": container with ID starting with 15ec0916fbe98ed135dbb6d88c278d14837f1af511a86f580caf1df178e8bac9 not found: ID does not exist"
Oct 30 18:55:30.430782 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:30.430756    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7fd531ef-8c6d-47dd-a840-ecc00a376373" path="/var/lib/kubelet/pods/7fd531ef-8c6d-47dd-a840-ecc00a376373/volumes"
Oct 30 18:55:30.431214 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:30.431196    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw" event={"ID":"c938f52d-8915-41a8-968d-253e99f1f490","Type":"ContainerStarted","Data":"bbdc83e6e370c3879c21097e10d698f4444448647409cfc99f6257e16ee205c4"}
Oct 30 18:55:30.441604 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:30.441560    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw" podStartSLOduration=1.980865584 podStartE2EDuration="2.441547625s" totalImagesPullingTime="460.682041ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:30.437739314 +0000 UTC m=+1152.154084391" watchObservedRunningTime="2026-10-30 18:55:30.441547625 +0000 UTC m=+1152.157892700"
Oct 30 18:55:31.065817 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:31.065686    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:31.425843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:31.425817    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:31.758295 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:31.758142    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:32.389544 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:32.388122    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:32.630935 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:32.630892    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:32.985896 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:32.985855    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:33.260194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:33.259692    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:33.454771 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:33.454744    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:33.878306 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:33.878249    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:35.497232 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:35.497199    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:35.677747 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:35.677716    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:36.052841 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:36.052801    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:36.329440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:36.329367    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"
Oct 30 18:55:36.737208 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:36.737172    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:37.015998 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:37.015908    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:37.157621 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:37.157588    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"]
Oct 30 18:55:37.308522 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:37.308424    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:37.336141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:37.336113    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"
Oct 30 18:55:37.437875 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:37.437851    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:55:37.501724 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:37.501624    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:38.038269 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.038234    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:38.047551 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.047524    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"]
Oct 30 18:55:38.052275 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.052257    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"
Oct 30 18:55:38.058904 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.058854    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-93574a-dockercfg-xd82k\""
Oct 30 18:55:38.059179 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.059163    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-93574a\""
Oct 30 18:55:38.062448 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.062425    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"]
Oct 30 18:55:38.065609 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.065585    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cxx8\" (UniqueName: \"kubernetes.io/projected/96d7b411-2194-4163-8f9e-ec82e15011a5-kube-api-access-5cxx8\") pod \"payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms\" (UID: \"96d7b411-2194-4163-8f9e-ec82e15011a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"
Oct 30 18:55:38.065791 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.065763    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96d7b411-2194-4163-8f9e-ec82e15011a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms\" (UID: \"96d7b411-2194-4163-8f9e-ec82e15011a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"
Oct 30 18:55:38.105151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.105122    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"]
Oct 30 18:55:38.109920 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.109900    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"
Oct 30 18:55:38.118668 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.118645    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"]
Oct 30 18:55:38.168032 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.167993    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cxx8\" (UniqueName: \"kubernetes.io/projected/96d7b411-2194-4163-8f9e-ec82e15011a5-kube-api-access-5cxx8\") pod \"payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms\" (UID: \"96d7b411-2194-4163-8f9e-ec82e15011a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"
Oct 30 18:55:38.168425 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.168397    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96d7b411-2194-4163-8f9e-ec82e15011a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms\" (UID: \"96d7b411-2194-4163-8f9e-ec82e15011a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"
Oct 30 18:55:38.170017 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.169915    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96d7b411-2194-4163-8f9e-ec82e15011a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms\" (UID: \"96d7b411-2194-4163-8f9e-ec82e15011a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"
Oct 30 18:55:38.293990 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.293909    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:38.297149 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.277127    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8fc4\" (UniqueName: \"kubernetes.io/projected/115327e6-4ff9-44d4-8b4d-52e512b78a66-kube-api-access-f8fc4\") pod \"payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v\" (UID: \"115327e6-4ff9-44d4-8b4d-52e512b78a66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"
Oct 30 18:55:38.310268 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.297461    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/115327e6-4ff9-44d4-8b4d-52e512b78a66-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v\" (UID: \"115327e6-4ff9-44d4-8b4d-52e512b78a66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"
Oct 30 18:55:38.461969 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.460171    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cxx8\" (UniqueName: \"kubernetes.io/projected/96d7b411-2194-4163-8f9e-ec82e15011a5-kube-api-access-5cxx8\") pod \"payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms\" (UID: \"96d7b411-2194-4163-8f9e-ec82e15011a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"
Oct 30 18:55:38.464970 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.464700    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/115327e6-4ff9-44d4-8b4d-52e512b78a66-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v\" (UID: \"115327e6-4ff9-44d4-8b4d-52e512b78a66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"
Oct 30 18:55:38.465121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.465026    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8fc4\" (UniqueName: \"kubernetes.io/projected/115327e6-4ff9-44d4-8b4d-52e512b78a66-kube-api-access-f8fc4\") pod \"payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v\" (UID: \"115327e6-4ff9-44d4-8b4d-52e512b78a66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"
Oct 30 18:55:38.466320 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.466300    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"
Oct 30 18:55:38.469773 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.467850    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/115327e6-4ff9-44d4-8b4d-52e512b78a66-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v\" (UID: \"115327e6-4ff9-44d4-8b4d-52e512b78a66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"
Oct 30 18:55:38.484442 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.484383    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8fc4\" (UniqueName: \"kubernetes.io/projected/115327e6-4ff9-44d4-8b4d-52e512b78a66-kube-api-access-f8fc4\") pod \"payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v\" (UID: \"115327e6-4ff9-44d4-8b4d-52e512b78a66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"
Oct 30 18:55:38.572515 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.572458    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:38.735159 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.735130    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"
Oct 30 18:55:38.765716 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.764047    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"]
Oct 30 18:55:38.993359 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:38.993328    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:39.036314 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:39.036290    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"]
Oct 30 18:55:39.037809 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:55:39.037778    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115327e6_4ff9_44d4_8b4d_52e512b78a66.slice/crio-c5e8c071d03233564b4a4e98b5948b645d7a45fb1fcee1f18522d579daeb8335 WatchSource:0}: Error finding container c5e8c071d03233564b4a4e98b5948b645d7a45fb1fcee1f18522d579daeb8335: Status 404 returned error can't find the container with id c5e8c071d03233564b4a4e98b5948b645d7a45fb1fcee1f18522d579daeb8335
Oct 30 18:55:39.238128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:39.225462    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:39.589780 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:39.589740    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms" event={"ID":"96d7b411-2194-4163-8f9e-ec82e15011a5","Type":"ContainerStarted","Data":"96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f"}
Oct 30 18:55:39.589965 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:39.589812    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"
Oct 30 18:55:39.589965 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:39.589836    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms" event={"ID":"96d7b411-2194-4163-8f9e-ec82e15011a5","Type":"ContainerStarted","Data":"2f9c3c853ec7c5abd4e747f40918d5f31b762d9203dcdf79be407a9bacacb5e3"}
Oct 30 18:55:39.591509 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:39.591482    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v" event={"ID":"115327e6-4ff9-44d4-8b4d-52e512b78a66","Type":"ContainerStarted","Data":"319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f"}
Oct 30 18:55:39.591656 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:39.591530    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v" event={"ID":"115327e6-4ff9-44d4-8b4d-52e512b78a66","Type":"ContainerStarted","Data":"c5e8c071d03233564b4a4e98b5948b645d7a45fb1fcee1f18522d579daeb8335"}
Oct 30 18:55:39.591656 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:39.591559    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"
Oct 30 18:55:39.608105 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:39.608054    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms" podStartSLOduration=1.6080405 podStartE2EDuration="1.6080405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:39.605351831 +0000 UTC m=+1161.321696909" watchObservedRunningTime="2026-10-30 18:55:39.6080405 +0000 UTC m=+1161.324385575"
Oct 30 18:55:39.623969 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:39.623926    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v" podStartSLOduration=1.623910983 podStartE2EDuration="1.623910983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:39.620647299 +0000 UTC m=+1161.336992396" watchObservedRunningTime="2026-10-30 18:55:39.623910983 +0000 UTC m=+1161.340256059"
Oct 30 18:55:40.107118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:40.106490    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:40.720382 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:40.720351    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:41.368129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:41.366353    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:42.158174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:42.158137    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj" podUID="ed2c0474-08cf-47f7-a431-a35864c221d2" containerName="payload-processing" containerID="cri-o://88f8210f6788c8106a0aa3c33b364abd447a073691c3288228dce2ffaf5a81a9" gracePeriod=25
Oct 30 18:55:42.348758 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:42.348594    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:43.010594 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.010560    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:43.186122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.181758    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"]
Oct 30 18:55:43.188162 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.188070    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"]
Oct 30 18:55:43.188475 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.188458    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.193781 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.193762    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-93574a-serving-cert\""
Oct 30 18:55:43.273930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.273840    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc1c28a-abd2-4def-a6d7-201732ee1eee-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.273930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.273917    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-metrics-certs\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.274141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.273984    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljq5b\" (UniqueName: \"kubernetes.io/projected/afc1c28a-abd2-4def-a6d7-201732ee1eee-kube-api-access-ljq5b\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.274141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.274076    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-maas-api-tls\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.312006 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:55:43.311960    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:55:43.363940 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.363906    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:43.375649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.375620    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc1c28a-abd2-4def-a6d7-201732ee1eee-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.375812 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.375680    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-metrics-certs\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.375812 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.375762    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljq5b\" (UniqueName: \"kubernetes.io/projected/afc1c28a-abd2-4def-a6d7-201732ee1eee-kube-api-access-ljq5b\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.375927 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.375860    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-maas-api-tls\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.376087 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:55:43.376059    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-serving-cert: secret "maas-api-e2e-ipp-a-93574a-serving-cert" not found
Oct 30 18:55:43.376250 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:55:43.376148    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-maas-api-tls podName:afc1c28a-abd2-4def-a6d7-201732ee1eee nodeName:}" failed. No retries permitted until 2026-10-30 18:55:43.876129109 +0000 UTC m=+1165.592474179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-maas-api-tls") pod "maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f" (UID: "afc1c28a-abd2-4def-a6d7-201732ee1eee") : secret "maas-api-e2e-ipp-a-93574a-serving-cert" not found
Oct 30 18:55:43.376467 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.376440    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc1c28a-abd2-4def-a6d7-201732ee1eee-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.378489 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.378468    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-metrics-certs\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.393921 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.393894    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljq5b\" (UniqueName: \"kubernetes.io/projected/afc1c28a-abd2-4def-a6d7-201732ee1eee-kube-api-access-ljq5b\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.583712 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.583624    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:43.878278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.878197    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:43.903840 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.903801    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-maas-api-tls\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:43.906560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:43.906536    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-maas-api-tls\") pod \"maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:44.119013 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.118980    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:44.528788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.528757    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"]
Oct 30 18:55:44.529183 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:55:44.529146    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc1c28a_abd2_4def_a6d7_201732ee1eee.slice/crio-593a333c714f66699771305faab6b39bd49d507eff9194c3c3e86f0242497c5e WatchSource:0}: Error finding container 593a333c714f66699771305faab6b39bd49d507eff9194c3c3e86f0242497c5e: Status 404 returned error can't find the container with id 593a333c714f66699771305faab6b39bd49d507eff9194c3c3e86f0242497c5e
Oct 30 18:55:44.673026 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.672946    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:44.674307 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.674223    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f" event={"ID":"afc1c28a-abd2-4def-a6d7-201732ee1eee","Type":"ContainerStarted","Data":"593a333c714f66699771305faab6b39bd49d507eff9194c3c3e86f0242497c5e"}
Oct 30 18:55:44.842077 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.841989    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"]
Oct 30 18:55:44.847784 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.847761    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"
Oct 30 18:55:44.852183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.852164    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-040d9c\""
Oct 30 18:55:44.853432 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.853411    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-040d9c-dockercfg-rwfnv\""
Oct 30 18:55:44.855747 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.855726    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"]
Oct 30 18:55:44.874626 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.874597    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:44.890959 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.890930    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"]
Oct 30 18:55:44.910886 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.910859    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"
Oct 30 18:55:44.977676 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:44.977636    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"]
Oct 30 18:55:45.126278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.049062    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwbws\" (UniqueName: \"kubernetes.io/projected/10a7d0d8-8693-49e9-8c3a-c79efb533794-kube-api-access-mwbws\") pod \"payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs\" (UID: \"10a7d0d8-8693-49e9-8c3a-c79efb533794\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"
Oct 30 18:55:45.126553 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.126529    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a7d0d8-8693-49e9-8c3a-c79efb533794-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs\" (UID: \"10a7d0d8-8693-49e9-8c3a-c79efb533794\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"
Oct 30 18:55:45.126743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.126722    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afd042f4-726b-429f-98dc-7bdbdeafa6f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv\" (UID: \"afd042f4-726b-429f-98dc-7bdbdeafa6f7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"
Oct 30 18:55:45.145331 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.145292    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96tfn\" (UniqueName: \"kubernetes.io/projected/afd042f4-726b-429f-98dc-7bdbdeafa6f7-kube-api-access-96tfn\") pod \"payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv\" (UID: \"afd042f4-726b-429f-98dc-7bdbdeafa6f7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"
Oct 30 18:55:45.196811 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.147874    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:45.262715 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.262681    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96tfn\" (UniqueName: \"kubernetes.io/projected/afd042f4-726b-429f-98dc-7bdbdeafa6f7-kube-api-access-96tfn\") pod \"payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv\" (UID: \"afd042f4-726b-429f-98dc-7bdbdeafa6f7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"
Oct 30 18:55:45.262962 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.262936    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwbws\" (UniqueName: \"kubernetes.io/projected/10a7d0d8-8693-49e9-8c3a-c79efb533794-kube-api-access-mwbws\") pod \"payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs\" (UID: \"10a7d0d8-8693-49e9-8c3a-c79efb533794\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"
Oct 30 18:55:45.263050 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.262988    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a7d0d8-8693-49e9-8c3a-c79efb533794-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs\" (UID: \"10a7d0d8-8693-49e9-8c3a-c79efb533794\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"
Oct 30 18:55:45.263173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.263054    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afd042f4-726b-429f-98dc-7bdbdeafa6f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv\" (UID: \"afd042f4-726b-429f-98dc-7bdbdeafa6f7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"
Oct 30 18:55:45.263961 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.263937    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afd042f4-726b-429f-98dc-7bdbdeafa6f7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv\" (UID: \"afd042f4-726b-429f-98dc-7bdbdeafa6f7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"
Oct 30 18:55:45.265231 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.265203    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a7d0d8-8693-49e9-8c3a-c79efb533794-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs\" (UID: \"10a7d0d8-8693-49e9-8c3a-c79efb533794\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"
Oct 30 18:55:45.273963 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.273936    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96tfn\" (UniqueName: \"kubernetes.io/projected/afd042f4-726b-429f-98dc-7bdbdeafa6f7-kube-api-access-96tfn\") pod \"payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv\" (UID: \"afd042f4-726b-429f-98dc-7bdbdeafa6f7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"
Oct 30 18:55:45.274504 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.274482    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwbws\" (UniqueName: \"kubernetes.io/projected/10a7d0d8-8693-49e9-8c3a-c79efb533794-kube-api-access-mwbws\") pod \"payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs\" (UID: \"10a7d0d8-8693-49e9-8c3a-c79efb533794\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"
Oct 30 18:55:45.465770 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.465738    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"
Oct 30 18:55:45.529817 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.529783    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"
Oct 30 18:55:45.732265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.708153    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:45.767531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.767487    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"]
Oct 30 18:55:45.770548 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:55:45.770518    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd042f4_726b_429f_98dc_7bdbdeafa6f7.slice/crio-87966e82b4e370a4110fd8974e4de568e04c9ad9020c0595ea1d946b4964e591 WatchSource:0}: Error finding container 87966e82b4e370a4110fd8974e4de568e04c9ad9020c0595ea1d946b4964e591: Status 404 returned error can't find the container with id 87966e82b4e370a4110fd8974e4de568e04c9ad9020c0595ea1d946b4964e591
Oct 30 18:55:45.770901 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.770871    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f" event={"ID":"afc1c28a-abd2-4def-a6d7-201732ee1eee","Type":"ContainerStarted","Data":"7d19164f7c7c6b3bcd5ce69664ac0775acf77bd4e64f5e8495905a236b49083f"}
Oct 30 18:55:45.771229 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.771201    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:45.794720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.794672    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f" podStartSLOduration=2.075067466 podStartE2EDuration="2.79464186s" totalImagesPullingTime="719.574394ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:45.790166267 +0000 UTC m=+1167.506511344" watchObservedRunningTime="2026-10-30 18:55:45.79464186 +0000 UTC m=+1167.510986937"
Oct 30 18:55:45.842349 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:45.842326    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"]
Oct 30 18:55:45.844399 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:55:45.844372    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a7d0d8_8693_49e9_8c3a_c79efb533794.slice/crio-35b6fbed482868bf007330883e8068e15241f58497d9fe2ca1b3b20022374152 WatchSource:0}: Error finding container 35b6fbed482868bf007330883e8068e15241f58497d9fe2ca1b3b20022374152: Status 404 returned error can't find the container with id 35b6fbed482868bf007330883e8068e15241f58497d9fe2ca1b3b20022374152
Oct 30 18:55:46.518395 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:46.518363    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:46.779531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:46.779425    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs" event={"ID":"10a7d0d8-8693-49e9-8c3a-c79efb533794","Type":"ContainerStarted","Data":"c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804"}
Oct 30 18:55:46.779531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:46.779494    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs" event={"ID":"10a7d0d8-8693-49e9-8c3a-c79efb533794","Type":"ContainerStarted","Data":"35b6fbed482868bf007330883e8068e15241f58497d9fe2ca1b3b20022374152"}
Oct 30 18:55:46.780044 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:46.779594    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"
Oct 30 18:55:46.781444 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:46.781417    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv" event={"ID":"afd042f4-726b-429f-98dc-7bdbdeafa6f7","Type":"ContainerStarted","Data":"5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b"}
Oct 30 18:55:46.781570 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:46.781476    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv" event={"ID":"afd042f4-726b-429f-98dc-7bdbdeafa6f7","Type":"ContainerStarted","Data":"87966e82b4e370a4110fd8974e4de568e04c9ad9020c0595ea1d946b4964e591"}
Oct 30 18:55:46.781570 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:46.781552    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"
Oct 30 18:55:46.800473 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:46.800432    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs" podStartSLOduration=2.800420448 podStartE2EDuration="2.800420448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:46.795453161 +0000 UTC m=+1168.511798236" watchObservedRunningTime="2026-10-30 18:55:46.800420448 +0000 UTC m=+1168.516765554"
Oct 30 18:55:46.814079 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:46.814038    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv" podStartSLOduration=2.8140241919999998 podStartE2EDuration="2.814024192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:46.812188736 +0000 UTC m=+1168.528533841" watchObservedRunningTime="2026-10-30 18:55:46.814024192 +0000 UTC m=+1168.530369267"
Oct 30 18:55:47.207942 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:47.207903    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj" podUID="ed2c0474-08cf-47f7-a431-a35864c221d2" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 18:55:47.297132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:47.292338    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:47.807768 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:47.807736    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:48.328497 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:48.328443    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:49.004146 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:48.998333    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:49.938122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:49.936136    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:50.654423 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:50.654390    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"]
Oct 30 18:55:50.861953 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:50.861851    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:50.890324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:50.864335    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"
Oct 30 18:55:50.890719 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:50.890696    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"
Oct 30 18:55:50.890899 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:50.890880    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"]
Oct 30 18:55:50.891994 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:50.865403    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:50.944189 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:50.938982    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-040d9c-serving-cert\""
Oct 30 18:55:50.990443 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:50.990403    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx7tg\" (UniqueName: \"kubernetes.io/projected/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-kube-api-access-bx7tg\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:50.990610 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:50.990476    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:50.990610 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:50.990515    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-maas-api-tls\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:50.990766 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:50.990645    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-metrics-certs\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:51.104769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.099419    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx7tg\" (UniqueName: \"kubernetes.io/projected/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-kube-api-access-bx7tg\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:51.104769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.099505    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:51.104769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.099550    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-maas-api-tls\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:51.104769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.099698    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-metrics-certs\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:51.104769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.100967    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:51.104769 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:55:51.101456    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-serving-cert: secret "maas-api-e2e-ipp-a-040d9c-serving-cert" not found
Oct 30 18:55:51.104769 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:55:51.101558    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-maas-api-tls podName:69c1f490-4b34-47d5-bae8-b7e6c3bb67ae nodeName:}" failed. No retries permitted until 2026-10-30 18:55:51.601529195 +0000 UTC m=+1173.317874254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-maas-api-tls") pod "maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5" (UID: "69c1f490-4b34-47d5-bae8-b7e6c3bb67ae") : secret "maas-api-e2e-ipp-a-040d9c-serving-cert" not found
Oct 30 18:55:51.123118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.119038    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:51.192195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.184802    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-metrics-certs\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:51.192195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.186888    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx7tg\" (UniqueName: \"kubernetes.io/projected/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-kube-api-access-bx7tg\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:51.418827 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.418794    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:51.605647 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.605610    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-maas-api-tls\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:51.608526 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.608493    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-maas-api-tls\") pod \"maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:51.794255 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.794160    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:55:51.903489 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:51.903456    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:52.032412 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:52.025766    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:52.239910 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:52.239881    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"]
Oct 30 18:55:52.240422 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:55:52.240391    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69c1f490_4b34_47d5_bae8_b7e6c3bb67ae.slice/crio-1fe137af585cf153fb13cf6dbbe873e020ce13288d3c5dd0ef13a0553bea9c71 WatchSource:0}: Error finding container 1fe137af585cf153fb13cf6dbbe873e020ce13288d3c5dd0ef13a0553bea9c71: Status 404 returned error can't find the container with id 1fe137af585cf153fb13cf6dbbe873e020ce13288d3c5dd0ef13a0553bea9c71
Oct 30 18:55:52.681119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:52.676863    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:53.007278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:53.007190    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5" event={"ID":"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae","Type":"ContainerStarted","Data":"b662941d8439580fb1fdedca49923668374c2c30c67463480c10aebd17ebcb67"}
Oct 30 18:55:53.007278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:53.007252    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5" event={"ID":"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae","Type":"ContainerStarted","Data":"1fe137af585cf153fb13cf6dbbe873e020ce13288d3c5dd0ef13a0553bea9c71"}
Oct 30 18:55:53.007702 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:53.007400    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:53.028307 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:53.028263    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5" podStartSLOduration=2.519068493 podStartE2EDuration="3.028250028s" totalImagesPullingTime="509.181535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:53.025439424 +0000 UTC m=+1174.741784511" watchObservedRunningTime="2026-10-30 18:55:53.028250028 +0000 UTC m=+1174.744595100"
Oct 30 18:55:53.622243 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:53.622147    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:54.392207 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:54.391648    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:54.746889 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:54.746839    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"]
Oct 30 18:55:54.747537 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:54.747075    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv" podUID="a8545208-0e08-4181-9dc6-b498a6ed0ae1" containerName="maas-api" containerID="cri-o://54def9e6448efc7bb3d406db8ea26e07a70466b19a45682db3b90529de519409" gracePeriod=30
Oct 30 18:55:54.879728 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:54.879620    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:55.200118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.190622    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:55.226529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.226491    4925 generic.go:356] "Generic (PLEG): container finished" podID="a8545208-0e08-4181-9dc6-b498a6ed0ae1" containerID="54def9e6448efc7bb3d406db8ea26e07a70466b19a45682db3b90529de519409" exitCode=0
Oct 30 18:55:55.226814 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.226790    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv" event={"ID":"a8545208-0e08-4181-9dc6-b498a6ed0ae1","Type":"ContainerDied","Data":"54def9e6448efc7bb3d406db8ea26e07a70466b19a45682db3b90529de519409"}
Oct 30 18:55:55.355254 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.355221    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:55:55.491142 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.489918    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:55.550882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.550841    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-maas-api-tls\") pod \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") "
Oct 30 18:55:55.551077 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.551048    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-metrics-certs\") pod \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") "
Oct 30 18:55:55.551238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.551212    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8545208-0e08-4181-9dc6-b498a6ed0ae1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8545208-0e08-4181-9dc6-b498a6ed0ae1-openshift-service-ca\") pod \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") "
Oct 30 18:55:55.551362 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.551286    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8545208-0e08-4181-9dc6-b498a6ed0ae1-kube-api-access-w9glw\" (UniqueName: \"kubernetes.io/projected/a8545208-0e08-4181-9dc6-b498a6ed0ae1-kube-api-access-w9glw\") pod \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\" (UID: \"a8545208-0e08-4181-9dc6-b498a6ed0ae1\") "
Oct 30 18:55:55.551640 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.551614    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8545208-0e08-4181-9dc6-b498a6ed0ae1-openshift-service-ca" pod "a8545208-0e08-4181-9dc6-b498a6ed0ae1" (UID: "a8545208-0e08-4181-9dc6-b498a6ed0ae1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:55:55.552619 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.552601    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8545208-0e08-4181-9dc6-b498a6ed0ae1-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:55:55.553838 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.553811    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-maas-api-tls" pod "a8545208-0e08-4181-9dc6-b498a6ed0ae1" (UID: "a8545208-0e08-4181-9dc6-b498a6ed0ae1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:55:55.553908 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.553851    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8545208-0e08-4181-9dc6-b498a6ed0ae1-kube-api-access-w9glw" pod "a8545208-0e08-4181-9dc6-b498a6ed0ae1" (UID: "a8545208-0e08-4181-9dc6-b498a6ed0ae1"). InnerVolumeSpecName "kube-api-access-w9glw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:55:55.554016 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.554003    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-metrics-certs" pod "a8545208-0e08-4181-9dc6-b498a6ed0ae1" (UID: "a8545208-0e08-4181-9dc6-b498a6ed0ae1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:55:55.653816 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.653741    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:55:55.653816 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.653770    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9glw\" (UniqueName: \"kubernetes.io/projected/a8545208-0e08-4181-9dc6-b498a6ed0ae1-kube-api-access-w9glw\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:55:55.653816 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.653785    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a8545208-0e08-4181-9dc6-b498a6ed0ae1-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:55:55.695000 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:55.694966    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:56.319132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:56.305824    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:56.319132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:56.311999    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv" event={"ID":"a8545208-0e08-4181-9dc6-b498a6ed0ae1","Type":"ContainerDied","Data":"c06306d598378a16c1ae3994772fa90302b73b3a5e5ed270c8a41ca3ee566f36"}
Oct 30 18:55:56.319132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:56.312158    4925 scope.go:122] "RemoveContainer" containerID="54def9e6448efc7bb3d406db8ea26e07a70466b19a45682db3b90529de519409"
Oct 30 18:55:56.319132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:56.312407    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"
Oct 30 18:55:56.365145 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:56.365117    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"]
Oct 30 18:55:56.368453 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:56.368428    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv"]
Oct 30 18:55:56.422984 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:56.422956    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8545208-0e08-4181-9dc6-b498a6ed0ae1" path="/var/lib/kubelet/pods/a8545208-0e08-4181-9dc6-b498a6ed0ae1/volumes"
Oct 30 18:55:56.796478 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:56.796441    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"]
Oct 30 18:55:56.806780 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:56.806746    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"]
Oct 30 18:55:56.874135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:56.872028    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:57.095176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:57.086787    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:57.244250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:57.244214    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj" podUID="ed2c0474-08cf-47f7-a431-a35864c221d2" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 18:55:57.305121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:57.304127    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:57.778652 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:57.777045    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:57.806126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:57.805713    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"
Oct 30 18:55:57.808333 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:57.808313    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"
Oct 30 18:55:58.311306 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:58.311263    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-9phxv" podUID="a8545208-0e08-4181-9dc6-b498a6ed0ae1" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.92:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:58.655922 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:58.655060    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:58.915626 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:58.915592    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:59.021903 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:59.021796    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:55:59.133587 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:59.133546    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:59.178406 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:59.178318    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"]
Oct 30 18:55:59.188090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:59.188060    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"]
Oct 30 18:55:59.311060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:59.311026    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:59.531213 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:59.530697    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:59.787180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:55:59.786585    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:01.034732 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:01.034337    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:01.797585 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:01.797530    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v" podUID="115327e6-4ff9-44d4-8b4d-52e512b78a66" containerName="payload-processing" containerID="cri-o://319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f" gracePeriod=25
Oct 30 18:56:01.807463 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:01.807430    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms" podUID="96d7b411-2194-4163-8f9e-ec82e15011a5" containerName="payload-pre-processing" containerID="cri-o://96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f" gracePeriod=25
Oct 30 18:56:02.199497 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.198202    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:02.329204 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.329175    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"
Oct 30 18:56:02.394082 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.394047    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/115327e6-4ff9-44d4-8b4d-52e512b78a66-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/115327e6-4ff9-44d4-8b4d-52e512b78a66-plugins-config-volume\") pod \"115327e6-4ff9-44d4-8b4d-52e512b78a66\" (UID: \"115327e6-4ff9-44d4-8b4d-52e512b78a66\") "
Oct 30 18:56:02.394368 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.394348    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/115327e6-4ff9-44d4-8b4d-52e512b78a66-kube-api-access-f8fc4\" (UniqueName: \"kubernetes.io/projected/115327e6-4ff9-44d4-8b4d-52e512b78a66-kube-api-access-f8fc4\") pod \"115327e6-4ff9-44d4-8b4d-52e512b78a66\" (UID: \"115327e6-4ff9-44d4-8b4d-52e512b78a66\") "
Oct 30 18:56:02.395795 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.395765    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/115327e6-4ff9-44d4-8b4d-52e512b78a66-plugins-config-volume" pod "115327e6-4ff9-44d4-8b4d-52e512b78a66" (UID: "115327e6-4ff9-44d4-8b4d-52e512b78a66"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:02.410373 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.410343    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/115327e6-4ff9-44d4-8b4d-52e512b78a66-kube-api-access-f8fc4" pod "115327e6-4ff9-44d4-8b4d-52e512b78a66" (UID: "115327e6-4ff9-44d4-8b4d-52e512b78a66"). InnerVolumeSpecName "kube-api-access-f8fc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:02.416289 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.416262    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"
Oct 30 18:56:02.497868 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.497837    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96d7b411-2194-4163-8f9e-ec82e15011a5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96d7b411-2194-4163-8f9e-ec82e15011a5-plugins-config-volume\") pod \"96d7b411-2194-4163-8f9e-ec82e15011a5\" (UID: \"96d7b411-2194-4163-8f9e-ec82e15011a5\") "
Oct 30 18:56:02.498059 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.498018    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96d7b411-2194-4163-8f9e-ec82e15011a5-kube-api-access-5cxx8\" (UniqueName: \"kubernetes.io/projected/96d7b411-2194-4163-8f9e-ec82e15011a5-kube-api-access-5cxx8\") pod \"96d7b411-2194-4163-8f9e-ec82e15011a5\" (UID: \"96d7b411-2194-4163-8f9e-ec82e15011a5\") "
Oct 30 18:56:02.499082 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.498921    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/115327e6-4ff9-44d4-8b4d-52e512b78a66-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:02.499082 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.498959    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8fc4\" (UniqueName: \"kubernetes.io/projected/115327e6-4ff9-44d4-8b4d-52e512b78a66-kube-api-access-f8fc4\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:02.499884 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.499418    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96d7b411-2194-4163-8f9e-ec82e15011a5-plugins-config-volume" pod "96d7b411-2194-4163-8f9e-ec82e15011a5" (UID: "96d7b411-2194-4163-8f9e-ec82e15011a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:02.511283 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.511257    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96d7b411-2194-4163-8f9e-ec82e15011a5-kube-api-access-5cxx8" pod "96d7b411-2194-4163-8f9e-ec82e15011a5" (UID: "96d7b411-2194-4163-8f9e-ec82e15011a5"). InnerVolumeSpecName "kube-api-access-5cxx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:02.515543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.515516    4925 generic.go:356] "Generic (PLEG): container finished" podID="115327e6-4ff9-44d4-8b4d-52e512b78a66" containerID="319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f" exitCode=0
Oct 30 18:56:02.516466 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.515564    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"
Oct 30 18:56:02.516466 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.515582    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v" event={"ID":"115327e6-4ff9-44d4-8b4d-52e512b78a66","Type":"ContainerDied","Data":"319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f"}
Oct 30 18:56:02.516466 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.515863    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v" event={"ID":"115327e6-4ff9-44d4-8b4d-52e512b78a66","Type":"ContainerDied","Data":"c5e8c071d03233564b4a4e98b5948b645d7a45fb1fcee1f18522d579daeb8335"}
Oct 30 18:56:02.516466 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.515898    4925 scope.go:122] "RemoveContainer" containerID="319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f"
Oct 30 18:56:02.521606 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.521514    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:02.523210 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.523185    4925 generic.go:356] "Generic (PLEG): container finished" podID="ed2c0474-08cf-47f7-a431-a35864c221d2" containerID="88f8210f6788c8106a0aa3c33b364abd447a073691c3288228dce2ffaf5a81a9" exitCode=0
Oct 30 18:56:02.523435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.523411    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj" event={"ID":"ed2c0474-08cf-47f7-a431-a35864c221d2","Type":"ContainerDied","Data":"88f8210f6788c8106a0aa3c33b364abd447a073691c3288228dce2ffaf5a81a9"}
Oct 30 18:56:02.526349 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.526321    4925 generic.go:356] "Generic (PLEG): container finished" podID="96d7b411-2194-4163-8f9e-ec82e15011a5" containerID="96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f" exitCode=0
Oct 30 18:56:02.526462 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.526389    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms" event={"ID":"96d7b411-2194-4163-8f9e-ec82e15011a5","Type":"ContainerDied","Data":"96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f"}
Oct 30 18:56:02.526462 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.526431    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms" event={"ID":"96d7b411-2194-4163-8f9e-ec82e15011a5","Type":"ContainerDied","Data":"2f9c3c853ec7c5abd4e747f40918d5f31b762d9203dcdf79be407a9bacacb5e3"}
Oct 30 18:56:02.526600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.526508    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"
Oct 30 18:56:02.550743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.550719    4925 scope.go:122] "RemoveContainer" containerID="319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f"
Oct 30 18:56:02.551677 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:02.551647    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f\": container with ID starting with 319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f not found: ID does not exist" containerID="319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f"
Oct 30 18:56:02.551794 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.551686    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f"} err="failed to get container status \"319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f\": rpc error: code = NotFound desc = could not find container \"319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f\": container with ID starting with 319aab702b96175b7a0118cab15a84ecacc02d54515d6ab02c3de8a9afa7d67f not found: ID does not exist"
Oct 30 18:56:02.551794 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.551714    4925 scope.go:122] "RemoveContainer" containerID="96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f"
Oct 30 18:56:02.551794 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.551787    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"]
Oct 30 18:56:02.592199 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.556992    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-93574a-dddd744fd-9zb5v"]
Oct 30 18:56:02.600314 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.600217    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/96d7b411-2194-4163-8f9e-ec82e15011a5-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:02.600314 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.600237    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cxx8\" (UniqueName: \"kubernetes.io/projected/96d7b411-2194-4163-8f9e-ec82e15011a5-kube-api-access-5cxx8\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:02.600314 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.600273    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"]
Oct 30 18:56:02.607711 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.607670    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-93574a-689766c59-vppms"]
Oct 30 18:56:02.610381 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.610348    4925 scope.go:122] "RemoveContainer" containerID="96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f"
Oct 30 18:56:02.610758 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:02.610738    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f\": container with ID starting with 96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f not found: ID does not exist" containerID="96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f"
Oct 30 18:56:02.610758 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.610764    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f"} err="failed to get container status \"96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f\": rpc error: code = NotFound desc = could not find container \"96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f\": container with ID starting with 96b3527e27f4eb798918d44f2fd0b4f8249a676598e40cc91e9afd825bda7b9f not found: ID does not exist"
Oct 30 18:56:02.687773 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.687746    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"
Oct 30 18:56:02.712367 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.712294    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:02.804879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.803795    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed2c0474-08cf-47f7-a431-a35864c221d2-kube-api-access-frvdq\" (UniqueName: \"kubernetes.io/projected/ed2c0474-08cf-47f7-a431-a35864c221d2-kube-api-access-frvdq\") pod \"ed2c0474-08cf-47f7-a431-a35864c221d2\" (UID: \"ed2c0474-08cf-47f7-a431-a35864c221d2\") "
Oct 30 18:56:02.804879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.803885    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed2c0474-08cf-47f7-a431-a35864c221d2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed2c0474-08cf-47f7-a431-a35864c221d2-plugins-config-volume\") pod \"ed2c0474-08cf-47f7-a431-a35864c221d2\" (UID: \"ed2c0474-08cf-47f7-a431-a35864c221d2\") "
Oct 30 18:56:02.804879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.804641    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed2c0474-08cf-47f7-a431-a35864c221d2-plugins-config-volume" pod "ed2c0474-08cf-47f7-a431-a35864c221d2" (UID: "ed2c0474-08cf-47f7-a431-a35864c221d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:02.806911 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.806888    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed2c0474-08cf-47f7-a431-a35864c221d2-kube-api-access-frvdq" pod "ed2c0474-08cf-47f7-a431-a35864c221d2" (UID: "ed2c0474-08cf-47f7-a431-a35864c221d2"). InnerVolumeSpecName "kube-api-access-frvdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:02.905325 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.905295    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed2c0474-08cf-47f7-a431-a35864c221d2-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:02.905325 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.905324    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frvdq\" (UniqueName: \"kubernetes.io/projected/ed2c0474-08cf-47f7-a431-a35864c221d2-kube-api-access-frvdq\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:02.925858 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.925826    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:02.997633 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:02.997548    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"]
Oct 30 18:56:03.019111 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:03.019064    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"]
Oct 30 18:56:03.131467 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:03.131431    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:03.305635 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:03.305541    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:03.522559 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:03.522529    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:03.613773 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:03.613670    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj" event={"ID":"ed2c0474-08cf-47f7-a431-a35864c221d2","Type":"ContainerDied","Data":"2a7211b68cdfab4360cd12995f46a898667da074935cffbc52a7574efaf221fd"}
Oct 30 18:56:03.614019 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:03.614000    4925 scope.go:122] "RemoveContainer" containerID="88f8210f6788c8106a0aa3c33b364abd447a073691c3288228dce2ffaf5a81a9"
Oct 30 18:56:03.614298 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:03.614280    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"
Oct 30 18:56:03.703185 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:03.703150    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"]
Oct 30 18:56:03.714239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:03.714214    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-d7dnj"]
Oct 30 18:56:03.805114 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:03.805061    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:04.092543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.092513    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"]
Oct 30 18:56:04.093246 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.093231    4925 cpu_manager.go:414] "containerMap: removing container" podUID="ed2c0474-08cf-47f7-a431-a35864c221d2" containerName="payload-processing"
Oct 30 18:56:04.093295 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.093248    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed2c0474-08cf-47f7-a431-a35864c221d2" containerName="payload-processing"
Oct 30 18:56:04.093295 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.093274    4925 cpu_manager.go:414] "containerMap: removing container" podUID="96d7b411-2194-4163-8f9e-ec82e15011a5" containerName="payload-pre-processing"
Oct 30 18:56:04.093295 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.093279    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96d7b411-2194-4163-8f9e-ec82e15011a5" containerName="payload-pre-processing"
Oct 30 18:56:04.093392 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.093308    4925 cpu_manager.go:414] "containerMap: removing container" podUID="115327e6-4ff9-44d4-8b4d-52e512b78a66" containerName="payload-processing"
Oct 30 18:56:04.093392 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.093313    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="115327e6-4ff9-44d4-8b4d-52e512b78a66" containerName="payload-processing"
Oct 30 18:56:04.093392 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.093324    4925 cpu_manager.go:414] "containerMap: removing container" podUID="a8545208-0e08-4181-9dc6-b498a6ed0ae1" containerName="maas-api"
Oct 30 18:56:04.093392 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.093328    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8545208-0e08-4181-9dc6-b498a6ed0ae1" containerName="maas-api"
Oct 30 18:56:04.093523 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.093434    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="115327e6-4ff9-44d4-8b4d-52e512b78a66" containerName="payload-processing"
Oct 30 18:56:04.093523 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.093443    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8545208-0e08-4181-9dc6-b498a6ed0ae1" containerName="maas-api"
Oct 30 18:56:04.093523 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.093450    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed2c0474-08cf-47f7-a431-a35864c221d2" containerName="payload-processing"
Oct 30 18:56:04.093523 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.093460    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="96d7b411-2194-4163-8f9e-ec82e15011a5" containerName="payload-pre-processing"
Oct 30 18:56:04.098563 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.098543    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"
Oct 30 18:56:04.106205 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.106183    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-8f1c37\""
Oct 30 18:56:04.106359 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.106339    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-8f1c37-dockercfg-mk6vc\""
Oct 30 18:56:04.109647 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.109622    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"]
Oct 30 18:56:04.110321 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.110299    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:04.150275 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.150243    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/425dbdc1-8cf7-40b5-b019-c245612eed41-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825\" (UID: \"425dbdc1-8cf7-40b5-b019-c245612eed41\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"
Oct 30 18:56:04.150436 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.150412    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlfhb\" (UniqueName: \"kubernetes.io/projected/425dbdc1-8cf7-40b5-b019-c245612eed41-kube-api-access-rlfhb\") pod \"payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825\" (UID: \"425dbdc1-8cf7-40b5-b019-c245612eed41\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"
Oct 30 18:56:04.156062 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.156039    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"]
Oct 30 18:56:04.197134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.182180    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c" podUID="cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982" containerName="payload-processing" containerID="cri-o://7e4faa0adb98884aa245b2716938e26f12638420dc5881110049bf9638495b2f" gracePeriod=25
Oct 30 18:56:04.197134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.191513    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t" podUID="d258f72d-529b-46e6-9637-4f947f5c12b4" containerName="payload-pre-processing" containerID="cri-o://6c8e2c6a0925cabce79ea4c6ca2fd649f84f892226b6278c0cfaa68024af8b3d" gracePeriod=25
Oct 30 18:56:04.204295 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.204271    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"]
Oct 30 18:56:04.204529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.204512    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"
Oct 30 18:56:04.396119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.357391    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlfhb\" (UniqueName: \"kubernetes.io/projected/425dbdc1-8cf7-40b5-b019-c245612eed41-kube-api-access-rlfhb\") pod \"payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825\" (UID: \"425dbdc1-8cf7-40b5-b019-c245612eed41\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"
Oct 30 18:56:04.396119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.357616    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncdjb\" (UniqueName: \"kubernetes.io/projected/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-kube-api-access-ncdjb\") pod \"payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg\" (UID: \"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d\") " pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"
Oct 30 18:56:04.396119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.357692    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg\" (UID: \"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d\") " pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"
Oct 30 18:56:04.396119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.357769    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/425dbdc1-8cf7-40b5-b019-c245612eed41-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825\" (UID: \"425dbdc1-8cf7-40b5-b019-c245612eed41\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"
Oct 30 18:56:04.396119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.358695    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/425dbdc1-8cf7-40b5-b019-c245612eed41-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825\" (UID: \"425dbdc1-8cf7-40b5-b019-c245612eed41\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"
Oct 30 18:56:04.484121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.432858    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:04.484121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.472194    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncdjb\" (UniqueName: \"kubernetes.io/projected/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-kube-api-access-ncdjb\") pod \"payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg\" (UID: \"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d\") " pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"
Oct 30 18:56:04.484121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.472287    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg\" (UID: \"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d\") " pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"
Oct 30 18:56:04.484121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.473611    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-plugins-config-volume\") pod \"payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg\" (UID: \"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d\") " pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"
Oct 30 18:56:04.570513 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.555010    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlfhb\" (UniqueName: \"kubernetes.io/projected/425dbdc1-8cf7-40b5-b019-c245612eed41-kube-api-access-rlfhb\") pod \"payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825\" (UID: \"425dbdc1-8cf7-40b5-b019-c245612eed41\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"
Oct 30 18:56:04.592002 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.590669    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncdjb\" (UniqueName: \"kubernetes.io/projected/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-kube-api-access-ncdjb\") pod \"payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg\" (UID: \"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d\") " pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"
Oct 30 18:56:04.598670 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.598591    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="115327e6-4ff9-44d4-8b4d-52e512b78a66" path="/var/lib/kubelet/pods/115327e6-4ff9-44d4-8b4d-52e512b78a66/volumes"
Oct 30 18:56:04.599215 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.599195    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96d7b411-2194-4163-8f9e-ec82e15011a5" path="/var/lib/kubelet/pods/96d7b411-2194-4163-8f9e-ec82e15011a5/volumes"
Oct 30 18:56:04.599741 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.599710    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed2c0474-08cf-47f7-a431-a35864c221d2" path="/var/lib/kubelet/pods/ed2c0474-08cf-47f7-a431-a35864c221d2/volumes"
Oct 30 18:56:04.628922 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.628743    4925 generic.go:356] "Generic (PLEG): container finished" podID="d258f72d-529b-46e6-9637-4f947f5c12b4" containerID="6c8e2c6a0925cabce79ea4c6ca2fd649f84f892226b6278c0cfaa68024af8b3d" exitCode=0
Oct 30 18:56:04.628922 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.628871    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t" event={"ID":"d258f72d-529b-46e6-9637-4f947f5c12b4","Type":"ContainerDied","Data":"6c8e2c6a0925cabce79ea4c6ca2fd649f84f892226b6278c0cfaa68024af8b3d"}
Oct 30 18:56:04.631084 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.630855    4925 generic.go:356] "Generic (PLEG): container finished" podID="cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982" containerID="7e4faa0adb98884aa245b2716938e26f12638420dc5881110049bf9638495b2f" exitCode=0
Oct 30 18:56:04.631084 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.631012    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c" event={"ID":"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982","Type":"ContainerDied","Data":"7e4faa0adb98884aa245b2716938e26f12638420dc5881110049bf9638495b2f"}
Oct 30 18:56:04.719189 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.719164    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"
Oct 30 18:56:04.826372 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.826344    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"
Oct 30 18:56:04.854241 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.853935    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"
Oct 30 18:56:04.907222 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.906684    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d258f72d-529b-46e6-9637-4f947f5c12b4-kube-api-access-llrnc\" (UniqueName: \"kubernetes.io/projected/d258f72d-529b-46e6-9637-4f947f5c12b4-kube-api-access-llrnc\") pod \"d258f72d-529b-46e6-9637-4f947f5c12b4\" (UID: \"d258f72d-529b-46e6-9637-4f947f5c12b4\") "
Oct 30 18:56:04.907222 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.906948    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d258f72d-529b-46e6-9637-4f947f5c12b4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d258f72d-529b-46e6-9637-4f947f5c12b4-plugins-config-volume\") pod \"d258f72d-529b-46e6-9637-4f947f5c12b4\" (UID: \"d258f72d-529b-46e6-9637-4f947f5c12b4\") "
Oct 30 18:56:04.909232 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.909202    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d258f72d-529b-46e6-9637-4f947f5c12b4-plugins-config-volume" pod "d258f72d-529b-46e6-9637-4f947f5c12b4" (UID: "d258f72d-529b-46e6-9637-4f947f5c12b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:04.911323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.911281    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d258f72d-529b-46e6-9637-4f947f5c12b4-kube-api-access-llrnc" pod "d258f72d-529b-46e6-9637-4f947f5c12b4" (UID: "d258f72d-529b-46e6-9637-4f947f5c12b4"). InnerVolumeSpecName "kube-api-access-llrnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:04.912585 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.912477    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d258f72d-529b-46e6-9637-4f947f5c12b4-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:04.912585 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:04.912515    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llrnc\" (UniqueName: \"kubernetes.io/projected/d258f72d-529b-46e6-9637-4f947f5c12b4-kube-api-access-llrnc\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:05.046201 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.043533    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:05.121170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.114053    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"
Oct 30 18:56:05.149865 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.146247    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-kube-api-access-6kb6x\" (UniqueName: \"kubernetes.io/projected/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-kube-api-access-6kb6x\") pod \"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982\" (UID: \"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982\") "
Oct 30 18:56:05.149865 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.146321    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-plugins-config-volume\") pod \"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982\" (UID: \"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982\") "
Oct 30 18:56:05.149865 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.148907    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-plugins-config-volume" pod "cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982" (UID: "cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:05.205385 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.203553    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-kube-api-access-6kb6x" pod "cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982" (UID: "cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982"). InnerVolumeSpecName "kube-api-access-6kb6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:05.248484 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.248458    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kb6x\" (UniqueName: \"kubernetes.io/projected/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-kube-api-access-6kb6x\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:05.248773 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.248755    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:05.337887 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.337404    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"]
Oct 30 18:56:05.348825 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.348804    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"]
Oct 30 18:56:05.350890 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:56:05.350866    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09d5a17a_9dbb_4f29_b3d7_3a807e013a9d.slice/crio-7717ed193095d930cc78a3b8ac2ddbff1262a718fb0581fc8c776da73592b985 WatchSource:0}: Error finding container 7717ed193095d930cc78a3b8ac2ddbff1262a718fb0581fc8c776da73592b985: Status 404 returned error can't find the container with id 7717ed193095d930cc78a3b8ac2ddbff1262a718fb0581fc8c776da73592b985
Oct 30 18:56:05.642253 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.642175    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"
Oct 30 18:56:05.642674 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.642177    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t" event={"ID":"d258f72d-529b-46e6-9637-4f947f5c12b4","Type":"ContainerDied","Data":"f26198853bc8864a6f2806a32c8b48e25e50db7cc30ba7eaa59ec93efa867d34"}
Oct 30 18:56:05.642674 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.642312    4925 scope.go:122] "RemoveContainer" containerID="6c8e2c6a0925cabce79ea4c6ca2fd649f84f892226b6278c0cfaa68024af8b3d"
Oct 30 18:56:05.645412 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.645379    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg" event={"ID":"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d","Type":"ContainerStarted","Data":"d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c"}
Oct 30 18:56:05.645673 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.645438    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg" event={"ID":"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d","Type":"ContainerStarted","Data":"7717ed193095d930cc78a3b8ac2ddbff1262a718fb0581fc8c776da73592b985"}
Oct 30 18:56:05.645673 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.645470    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"
Oct 30 18:56:05.646919 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.646903    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"
Oct 30 18:56:05.647221 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.647165    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c" event={"ID":"cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982","Type":"ContainerDied","Data":"07c627cdc8ac31b22a6fc3f4fc91b846bdd9678cb028a79855e31f0599ab6ce0"}
Oct 30 18:56:05.649168 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.649145    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825" event={"ID":"425dbdc1-8cf7-40b5-b019-c245612eed41","Type":"ContainerStarted","Data":"bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f"}
Oct 30 18:56:05.649259 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.649190    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825" event={"ID":"425dbdc1-8cf7-40b5-b019-c245612eed41","Type":"ContainerStarted","Data":"587bf527fc67e5ff1f952097f040c0b57792e6996e8ae302eebe6431eace4d64"}
Oct 30 18:56:05.649259 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.649221    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"
Oct 30 18:56:05.655618 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.655602    4925 scope.go:122] "RemoveContainer" containerID="7e4faa0adb98884aa245b2716938e26f12638420dc5881110049bf9638495b2f"
Oct 30 18:56:05.683035 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.682991    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg" podStartSLOduration=1.6829758670000001 podStartE2EDuration="1.682975867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:05.670945079 +0000 UTC m=+1187.387290155" watchObservedRunningTime="2026-10-30 18:56:05.682975867 +0000 UTC m=+1187.399320942"
Oct 30 18:56:05.717751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.717708    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"]
Oct 30 18:56:05.722155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.722130    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-dc0f87-84978868ff-72m7t"]
Oct 30 18:56:05.727947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.726920    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825" podStartSLOduration=1.726907782 podStartE2EDuration="1.726907782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:05.723998655 +0000 UTC m=+1187.440343732" watchObservedRunningTime="2026-10-30 18:56:05.726907782 +0000 UTC m=+1187.443252857"
Oct 30 18:56:05.744561 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.744528    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"]
Oct 30 18:56:05.748567 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.748543    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-dc0f87-54d6cc5b6b-zrd9c"]
Oct 30 18:56:05.909166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:05.909056    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:06.090322 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.090288    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:06.276947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.276798    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:06.424283 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.424252    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982" path="/var/lib/kubelet/pods/cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982/volumes"
Oct 30 18:56:06.424661 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.424646    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d258f72d-529b-46e6-9637-4f947f5c12b4" path="/var/lib/kubelet/pods/d258f72d-529b-46e6-9637-4f947f5c12b4/volumes"
Oct 30 18:56:06.465155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.465127    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"]
Oct 30 18:56:06.466204 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.466184    4925 cpu_manager.go:414] "containerMap: removing container" podUID="cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982" containerName="payload-processing"
Oct 30 18:56:06.466316 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.466205    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982" containerName="payload-processing"
Oct 30 18:56:06.466316 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.466228    4925 cpu_manager.go:414] "containerMap: removing container" podUID="d258f72d-529b-46e6-9637-4f947f5c12b4" containerName="payload-pre-processing"
Oct 30 18:56:06.466316 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.466237    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d258f72d-529b-46e6-9637-4f947f5c12b4" containerName="payload-pre-processing"
Oct 30 18:56:06.466473 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.466400    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="cbbc3b1e-bb21-4ca4-9007-d5a0d25e4982" containerName="payload-processing"
Oct 30 18:56:06.466473 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.466414    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="d258f72d-529b-46e6-9637-4f947f5c12b4" containerName="payload-pre-processing"
Oct 30 18:56:06.473490 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.473463    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"]
Oct 30 18:56:06.473624 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.473499    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"
Oct 30 18:56:06.478713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.478688    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"
Oct 30 18:56:06.493756 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.493733    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"]
Oct 30 18:56:06.494484 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.494463    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-da58b769\""
Oct 30 18:56:06.504220 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.503304    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-da58b769-dockercfg-dwbvz\""
Oct 30 18:56:06.512353 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.512277    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"]
Oct 30 18:56:06.625198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.617000    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb\" (UID: \"97ca9fe2-b463-4ade-be86-b4ffa7a34cef\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"
Oct 30 18:56:06.625198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.622133    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:06.645343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.643691    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7s9d\" (UniqueName: \"kubernetes.io/projected/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-kube-api-access-f7s9d\") pod \"payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb\" (UID: \"97ca9fe2-b463-4ade-be86-b4ffa7a34cef\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"
Oct 30 18:56:06.645343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.643825    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/595201e2-381f-488d-a987-cb741ac1254b-plugins-config-volume\") pod \"payload-processing-e2e-mt-da58b769-7ddb646744-dvj75\" (UID: \"595201e2-381f-488d-a987-cb741ac1254b\") " pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"
Oct 30 18:56:06.645343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.643921    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjbp5\" (UniqueName: \"kubernetes.io/projected/595201e2-381f-488d-a987-cb741ac1254b-kube-api-access-jjbp5\") pod \"payload-processing-e2e-mt-da58b769-7ddb646744-dvj75\" (UID: \"595201e2-381f-488d-a987-cb741ac1254b\") " pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"
Oct 30 18:56:06.750185 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.750123    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7s9d\" (UniqueName: \"kubernetes.io/projected/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-kube-api-access-f7s9d\") pod \"payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb\" (UID: \"97ca9fe2-b463-4ade-be86-b4ffa7a34cef\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"
Oct 30 18:56:06.751356 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.751326    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/595201e2-381f-488d-a987-cb741ac1254b-plugins-config-volume\") pod \"payload-processing-e2e-mt-da58b769-7ddb646744-dvj75\" (UID: \"595201e2-381f-488d-a987-cb741ac1254b\") " pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"
Oct 30 18:56:06.752849 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.752829    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjbp5\" (UniqueName: \"kubernetes.io/projected/595201e2-381f-488d-a987-cb741ac1254b-kube-api-access-jjbp5\") pod \"payload-processing-e2e-mt-da58b769-7ddb646744-dvj75\" (UID: \"595201e2-381f-488d-a987-cb741ac1254b\") " pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"
Oct 30 18:56:06.753330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.753312    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb\" (UID: \"97ca9fe2-b463-4ade-be86-b4ffa7a34cef\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"
Oct 30 18:56:06.754742 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.752709    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/595201e2-381f-488d-a987-cb741ac1254b-plugins-config-volume\") pod \"payload-processing-e2e-mt-da58b769-7ddb646744-dvj75\" (UID: \"595201e2-381f-488d-a987-cb741ac1254b\") " pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"
Oct 30 18:56:06.754866 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.754459    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb\" (UID: \"97ca9fe2-b463-4ade-be86-b4ffa7a34cef\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"
Oct 30 18:56:06.803703 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.798036    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7s9d\" (UniqueName: \"kubernetes.io/projected/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-kube-api-access-f7s9d\") pod \"payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb\" (UID: \"97ca9fe2-b463-4ade-be86-b4ffa7a34cef\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"
Oct 30 18:56:06.830120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:06.827005    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjbp5\" (UniqueName: \"kubernetes.io/projected/595201e2-381f-488d-a987-cb741ac1254b-kube-api-access-jjbp5\") pod \"payload-processing-e2e-mt-da58b769-7ddb646744-dvj75\" (UID: \"595201e2-381f-488d-a987-cb741ac1254b\") " pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"
Oct 30 18:56:07.096907 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.096868    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"
Oct 30 18:56:07.112054 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.112028    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"
Oct 30 18:56:07.372519 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.372145    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:07.416060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.414838    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"]
Oct 30 18:56:07.422349 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.422327    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"]
Oct 30 18:56:07.424197 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:56:07.424167    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595201e2_381f_488d_a987_cb741ac1254b.slice/crio-cf347ce5cc142ae829668528236302988a1d38294666d469f099a72e6253c04d WatchSource:0}: Error finding container cf347ce5cc142ae829668528236302988a1d38294666d469f099a72e6253c04d: Status 404 returned error can't find the container with id cf347ce5cc142ae829668528236302988a1d38294666d469f099a72e6253c04d
Oct 30 18:56:07.716417 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.716364    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75" event={"ID":"595201e2-381f-488d-a987-cb741ac1254b","Type":"ContainerStarted","Data":"d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00"}
Oct 30 18:56:07.716943 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.716446    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75" event={"ID":"595201e2-381f-488d-a987-cb741ac1254b","Type":"ContainerStarted","Data":"cf347ce5cc142ae829668528236302988a1d38294666d469f099a72e6253c04d"}
Oct 30 18:56:07.716943 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.716489    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"
Oct 30 18:56:07.718560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.718529    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb" event={"ID":"97ca9fe2-b463-4ade-be86-b4ffa7a34cef","Type":"ContainerStarted","Data":"b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e"}
Oct 30 18:56:07.718687 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.718588    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb" event={"ID":"97ca9fe2-b463-4ade-be86-b4ffa7a34cef","Type":"ContainerStarted","Data":"64f74f32683676398be829d6fc887ac163cd80a416e052274dc36952193a3302"}
Oct 30 18:56:07.718687 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.718621    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"
Oct 30 18:56:07.744836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.744774    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75" podStartSLOduration=1.744754847 podStartE2EDuration="1.744754847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:07.740743509 +0000 UTC m=+1189.457088582" watchObservedRunningTime="2026-10-30 18:56:07.744754847 +0000 UTC m=+1189.461099927"
Oct 30 18:56:07.759642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.759575    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb" podStartSLOduration=1.759558385 podStartE2EDuration="1.759558385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:07.75678373 +0000 UTC m=+1189.473128806" watchObservedRunningTime="2026-10-30 18:56:07.759558385 +0000 UTC m=+1189.475903462"
Oct 30 18:56:07.998771 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:07.998047    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs" podUID="10a7d0d8-8693-49e9-8c3a-c79efb533794" containerName="payload-processing" containerID="cri-o://c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804" gracePeriod=25
Oct 30 18:56:08.020046 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.019985    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv" podUID="afd042f4-726b-429f-98dc-7bdbdeafa6f7" containerName="payload-pre-processing" containerID="cri-o://5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b" gracePeriod=25
Oct 30 18:56:08.380630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.363437    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"
Oct 30 18:56:08.465064 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.464931    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:08.480711 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.480133    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"
Oct 30 18:56:08.512037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.511697    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10a7d0d8-8693-49e9-8c3a-c79efb533794-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a7d0d8-8693-49e9-8c3a-c79efb533794-plugins-config-volume\") pod \"10a7d0d8-8693-49e9-8c3a-c79efb533794\" (UID: \"10a7d0d8-8693-49e9-8c3a-c79efb533794\") "
Oct 30 18:56:08.512037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.511824    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10a7d0d8-8693-49e9-8c3a-c79efb533794-kube-api-access-mwbws\" (UniqueName: \"kubernetes.io/projected/10a7d0d8-8693-49e9-8c3a-c79efb533794-kube-api-access-mwbws\") pod \"10a7d0d8-8693-49e9-8c3a-c79efb533794\" (UID: \"10a7d0d8-8693-49e9-8c3a-c79efb533794\") "
Oct 30 18:56:08.519138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.518881    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10a7d0d8-8693-49e9-8c3a-c79efb533794-kube-api-access-mwbws" pod "10a7d0d8-8693-49e9-8c3a-c79efb533794" (UID: "10a7d0d8-8693-49e9-8c3a-c79efb533794"). InnerVolumeSpecName "kube-api-access-mwbws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:08.519855 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.519445    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10a7d0d8-8693-49e9-8c3a-c79efb533794-plugins-config-volume" pod "10a7d0d8-8693-49e9-8c3a-c79efb533794" (UID: "10a7d0d8-8693-49e9-8c3a-c79efb533794"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:08.622614 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.615811    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/afd042f4-726b-429f-98dc-7bdbdeafa6f7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afd042f4-726b-429f-98dc-7bdbdeafa6f7-plugins-config-volume\") pod \"afd042f4-726b-429f-98dc-7bdbdeafa6f7\" (UID: \"afd042f4-726b-429f-98dc-7bdbdeafa6f7\") "
Oct 30 18:56:08.631175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.625135    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afd042f4-726b-429f-98dc-7bdbdeafa6f7-plugins-config-volume" pod "afd042f4-726b-429f-98dc-7bdbdeafa6f7" (UID: "afd042f4-726b-429f-98dc-7bdbdeafa6f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:08.631175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.625198    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afd042f4-726b-429f-98dc-7bdbdeafa6f7-kube-api-access-96tfn\" (UniqueName: \"kubernetes.io/projected/afd042f4-726b-429f-98dc-7bdbdeafa6f7-kube-api-access-96tfn\") pod \"afd042f4-726b-429f-98dc-7bdbdeafa6f7\" (UID: \"afd042f4-726b-429f-98dc-7bdbdeafa6f7\") "
Oct 30 18:56:08.631175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.626050    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/10a7d0d8-8693-49e9-8c3a-c79efb533794-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:08.631175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.626078    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwbws\" (UniqueName: \"kubernetes.io/projected/10a7d0d8-8693-49e9-8c3a-c79efb533794-kube-api-access-mwbws\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:08.708697 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.696391    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afd042f4-726b-429f-98dc-7bdbdeafa6f7-kube-api-access-96tfn" pod "afd042f4-726b-429f-98dc-7bdbdeafa6f7" (UID: "afd042f4-726b-429f-98dc-7bdbdeafa6f7"). InnerVolumeSpecName "kube-api-access-96tfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:08.731210 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.730517    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96tfn\" (UniqueName: \"kubernetes.io/projected/afd042f4-726b-429f-98dc-7bdbdeafa6f7-kube-api-access-96tfn\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:08.731210 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.730558    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afd042f4-726b-429f-98dc-7bdbdeafa6f7-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:08.755871 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.755839    4925 generic.go:356] "Generic (PLEG): container finished" podID="10a7d0d8-8693-49e9-8c3a-c79efb533794" containerID="c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804" exitCode=0
Oct 30 18:56:08.756037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.756001    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs" event={"ID":"10a7d0d8-8693-49e9-8c3a-c79efb533794","Type":"ContainerDied","Data":"c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804"}
Oct 30 18:56:08.756088 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.756065    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs" event={"ID":"10a7d0d8-8693-49e9-8c3a-c79efb533794","Type":"ContainerDied","Data":"35b6fbed482868bf007330883e8068e15241f58497d9fe2ca1b3b20022374152"}
Oct 30 18:56:08.756088 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.756085    4925 scope.go:122] "RemoveContainer" containerID="c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804"
Oct 30 18:56:08.756378 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.756358    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"
Oct 30 18:56:08.761736 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.761708    4925 generic.go:356] "Generic (PLEG): container finished" podID="afd042f4-726b-429f-98dc-7bdbdeafa6f7" containerID="5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b" exitCode=0
Oct 30 18:56:08.762646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.762022    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv" event={"ID":"afd042f4-726b-429f-98dc-7bdbdeafa6f7","Type":"ContainerDied","Data":"5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b"}
Oct 30 18:56:08.762646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.762186    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv" event={"ID":"afd042f4-726b-429f-98dc-7bdbdeafa6f7","Type":"ContainerDied","Data":"87966e82b4e370a4110fd8974e4de568e04c9ad9020c0595ea1d946b4964e591"}
Oct 30 18:56:08.762646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.762602    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"
Oct 30 18:56:08.774074 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.774052    4925 scope.go:122] "RemoveContainer" containerID="c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804"
Oct 30 18:56:08.774583 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:08.774557    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804\": container with ID starting with c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804 not found: ID does not exist" containerID="c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804"
Oct 30 18:56:08.774672 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.774592    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804"} err="failed to get container status \"c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804\": rpc error: code = NotFound desc = could not find container \"c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804\": container with ID starting with c088cf30453aed0378d0dd26d920861f7d7bb95bb939c523eb3295110e452804 not found: ID does not exist"
Oct 30 18:56:08.774672 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.774615    4925 scope.go:122] "RemoveContainer" containerID="5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b"
Oct 30 18:56:08.787156 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.787136    4925 scope.go:122] "RemoveContainer" containerID="5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b"
Oct 30 18:56:08.787460 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:08.787434    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b\": container with ID starting with 5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b not found: ID does not exist" containerID="5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b"
Oct 30 18:56:08.787537 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.787469    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b"} err="failed to get container status \"5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b\": rpc error: code = NotFound desc = could not find container \"5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b\": container with ID starting with 5a81bd64db4737ee0bdb73931c6d8c9a71cb1e887e57217ce1a2dec6b8268d8b not found: ID does not exist"
Oct 30 18:56:08.798516 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.798495    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"]
Oct 30 18:56:08.804426 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.804403    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-040d9c-6f7b767444-gc4bs"]
Oct 30 18:56:08.809646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.809623    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"]
Oct 30 18:56:08.812623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.812587    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-7b448dc8c9-sxlvv"]
Oct 30 18:56:08.909904 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:08.909861    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:09.219543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:09.219331    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:09.878446 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:09.878412    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:10.425666 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.425636    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10a7d0d8-8693-49e9-8c3a-c79efb533794" path="/var/lib/kubelet/pods/10a7d0d8-8693-49e9-8c3a-c79efb533794/volumes"
Oct 30 18:56:10.426045 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.426032    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afd042f4-726b-429f-98dc-7bdbdeafa6f7" path="/var/lib/kubelet/pods/afd042f4-726b-429f-98dc-7bdbdeafa6f7/volumes"
Oct 30 18:56:10.426394 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.426379    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"]
Oct 30 18:56:10.427409 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.427386    4925 cpu_manager.go:414] "containerMap: removing container" podUID="afd042f4-726b-429f-98dc-7bdbdeafa6f7" containerName="payload-pre-processing"
Oct 30 18:56:10.427409 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.427406    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afd042f4-726b-429f-98dc-7bdbdeafa6f7" containerName="payload-pre-processing"
Oct 30 18:56:10.427607 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.427479    4925 cpu_manager.go:414] "containerMap: removing container" podUID="10a7d0d8-8693-49e9-8c3a-c79efb533794" containerName="payload-processing"
Oct 30 18:56:10.427607 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.427485    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10a7d0d8-8693-49e9-8c3a-c79efb533794" containerName="payload-processing"
Oct 30 18:56:10.427713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.427610    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="afd042f4-726b-429f-98dc-7bdbdeafa6f7" containerName="payload-pre-processing"
Oct 30 18:56:10.427713 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.427625    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="10a7d0d8-8693-49e9-8c3a-c79efb533794" containerName="payload-processing"
Oct 30 18:56:10.435949 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.435717    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:10.436276 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.436254    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"]
Oct 30 18:56:10.436408 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.436366    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"
Oct 30 18:56:10.444597 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.444575    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-040d9c-dockercfg-cb4bj\""
Oct 30 18:56:10.444847 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.444826    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-040d9c\""
Oct 30 18:56:10.449135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.445710    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"]
Oct 30 18:56:10.451444 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.451422    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"
Oct 30 18:56:10.472249 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.472215    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"]
Oct 30 18:56:10.602121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.597123    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v\" (UID: \"f29a9a95-cd18-4a60-a38b-c83b77cfadf2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"
Oct 30 18:56:10.602121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.597430    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2\" (UID: \"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"
Oct 30 18:56:10.602121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.597490    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjbsp\" (UniqueName: \"kubernetes.io/projected/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-kube-api-access-wjbsp\") pod \"payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2\" (UID: \"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"
Oct 30 18:56:10.602121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.597679    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg29z\" (UniqueName: \"kubernetes.io/projected/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-kube-api-access-hg29z\") pod \"payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v\" (UID: \"f29a9a95-cd18-4a60-a38b-c83b77cfadf2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"
Oct 30 18:56:10.651120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.643087    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:10.712169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.702267    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg29z\" (UniqueName: \"kubernetes.io/projected/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-kube-api-access-hg29z\") pod \"payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v\" (UID: \"f29a9a95-cd18-4a60-a38b-c83b77cfadf2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"
Oct 30 18:56:10.712169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.702372    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v\" (UID: \"f29a9a95-cd18-4a60-a38b-c83b77cfadf2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"
Oct 30 18:56:10.712169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.702628    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2\" (UID: \"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"
Oct 30 18:56:10.712169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.702679    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjbsp\" (UniqueName: \"kubernetes.io/projected/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-kube-api-access-wjbsp\") pod \"payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2\" (UID: \"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"
Oct 30 18:56:10.712169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.704202    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v\" (UID: \"f29a9a95-cd18-4a60-a38b-c83b77cfadf2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"
Oct 30 18:56:10.712169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.704935    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2\" (UID: \"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"
Oct 30 18:56:10.742120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.740204    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjbsp\" (UniqueName: \"kubernetes.io/projected/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-kube-api-access-wjbsp\") pod \"payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2\" (UID: \"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"
Oct 30 18:56:10.762119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.753946    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg29z\" (UniqueName: \"kubernetes.io/projected/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-kube-api-access-hg29z\") pod \"payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v\" (UID: \"f29a9a95-cd18-4a60-a38b-c83b77cfadf2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"
Oct 30 18:56:10.801606 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.801337    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"
Oct 30 18:56:10.803262 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:10.803237    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"
Oct 30 18:56:11.113175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:11.105366    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:11.160520 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:11.156436    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"]
Oct 30 18:56:11.162014 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:11.161988    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"]
Oct 30 18:56:11.181375 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:56:11.180263    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda1aecc_c6a3_4d7d_ad62_f78fb46264ba.slice/crio-e3033f3916b66c2760b052609f808d68e570e1559a0c802ec1c71134ff885988 WatchSource:0}: Error finding container e3033f3916b66c2760b052609f808d68e570e1559a0c802ec1c71134ff885988: Status 404 returned error can't find the container with id e3033f3916b66c2760b052609f808d68e570e1559a0c802ec1c71134ff885988
Oct 30 18:56:11.878333 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:11.878290    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v" event={"ID":"f29a9a95-cd18-4a60-a38b-c83b77cfadf2","Type":"ContainerStarted","Data":"5fc42657e921edb532d119aec7be84222817e70c8f0fdfac8b4bb3788419265e"}
Oct 30 18:56:11.882595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:11.881168    4925 generic.go:356] "Generic (PLEG): container finished" podID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerID="852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442" exitCode=0
Oct 30 18:56:11.882595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:11.881277    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" event={"ID":"086f5d84-6529-4e59-bbc9-ec0141cd16de","Type":"ContainerDied","Data":"852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442"}
Oct 30 18:56:11.882595 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:11.881752    4925 scope.go:122] "RemoveContainer" containerID="852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442"
Oct 30 18:56:11.885195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:11.884245    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:56:11.885195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:11.884330    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:56:11.885195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:11.884379    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:56:11.887353 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:11.887280    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2" event={"ID":"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba","Type":"ContainerStarted","Data":"e3033f3916b66c2760b052609f808d68e570e1559a0c802ec1c71134ff885988"}
Oct 30 18:56:11.957903 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:11.951261    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:12.145930 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:12.144282    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:12.610314 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:12.610268    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:56:12.965119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:12.959042    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:12.970943 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:12.970179    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" event={"ID":"086f5d84-6529-4e59-bbc9-ec0141cd16de","Type":"ContainerStarted","Data":"c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0"}
Oct 30 18:56:13.337750 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:13.337677    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:56:13.842536 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:13.842502    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:56:13.844345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:13.844301    4925 patch_prober.go:29] interesting pod/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 18:56:13.844528 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:13.844365    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 18:56:14.127171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:14.124347    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:14.847154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:14.843225    4925 patch_prober.go:29] interesting pod/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 18:56:14.847154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:14.843293    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 18:56:15.406117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:15.404638    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:15.843071 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:15.843036    4925 patch_prober.go:29] interesting pod/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 18:56:15.843260 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:15.843109    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 18:56:16.294693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:16.294649    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:16.717116 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:16.712514    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"
Oct 30 18:56:16.717116 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:16.713653    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"
Oct 30 18:56:16.843764 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:16.843728    4925 patch_prober.go:29] interesting pod/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 18:56:16.843956 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:16.843786    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 18:56:17.016831 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.016734    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"]
Oct 30 18:56:17.029352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.028280    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"]
Oct 30 18:56:17.029352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.028398    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.032811 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.032787    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-da58b769-serving-cert\""
Oct 30 18:56:17.056079 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.056050    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v" event={"ID":"f29a9a95-cd18-4a60-a38b-c83b77cfadf2","Type":"ContainerStarted","Data":"935edee7c27ce5b6879b9654bd194103f6568a7ad497e104d9e66ff33fc8247e"}
Oct 30 18:56:17.085125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.057148    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"
Oct 30 18:56:17.108691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.108651    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2" event={"ID":"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba","Type":"ContainerStarted","Data":"f6f4b8d70c614840380e2e6fa361d461bde0e51c30be60b6c7a4a66c795cbad4"}
Oct 30 18:56:17.113249 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.113229    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"
Oct 30 18:56:17.225124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.156188    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:17.225124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.185575    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e573e700-5f11-4a17-9318-6d89ecbce82f-openshift-service-ca\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.225124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.185721    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-maas-api-tls\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.225124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.185815    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvfnz\" (UniqueName: \"kubernetes.io/projected/e573e700-5f11-4a17-9318-6d89ecbce82f-kube-api-access-tvfnz\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.225124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.186040    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-metrics-certs\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.283176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.263699    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v" podStartSLOduration=2.477847316 podStartE2EDuration="7.263677835s" totalImagesPullingTime="4.785830519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:17.155389741 +0000 UTC m=+1198.871734817" watchObservedRunningTime="2026-10-30 18:56:17.263677835 +0000 UTC m=+1198.980022911"
Oct 30 18:56:17.338836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.321246    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-metrics-certs\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.338836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.334643    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e573e700-5f11-4a17-9318-6d89ecbce82f-openshift-service-ca\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.338836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.334823    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-maas-api-tls\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.338836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.334912    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvfnz\" (UniqueName: \"kubernetes.io/projected/e573e700-5f11-4a17-9318-6d89ecbce82f-kube-api-access-tvfnz\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.338836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.337121    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e573e700-5f11-4a17-9318-6d89ecbce82f-openshift-service-ca\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.338836 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:17.337234    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-serving-cert: secret "maas-api-e2e-mt-da58b769-serving-cert" not found
Oct 30 18:56:17.338836 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:17.337302    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-maas-api-tls podName:e573e700-5f11-4a17-9318-6d89ecbce82f nodeName:}" failed. No retries permitted until 2026-10-30 18:56:17.837282567 +0000 UTC m=+1199.553627640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-maas-api-tls") pod "maas-api-e2e-mt-da58b769-c58747d4f-fc5b4" (UID: "e573e700-5f11-4a17-9318-6d89ecbce82f") : secret "maas-api-e2e-mt-da58b769-serving-cert" not found
Oct 30 18:56:17.369991 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.369958    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:17.389323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.389290    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-metrics-certs\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.390374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.390317    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2" podStartSLOduration=2.652649264 podStartE2EDuration="7.390302622s" totalImagesPullingTime="4.737653358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:17.183693929 +0000 UTC m=+1198.900039006" watchObservedRunningTime="2026-10-30 18:56:17.390302622 +0000 UTC m=+1199.106647699"
Oct 30 18:56:17.411120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.404786    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvfnz\" (UniqueName: \"kubernetes.io/projected/e573e700-5f11-4a17-9318-6d89ecbce82f-kube-api-access-tvfnz\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.844530 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.844489    4925 patch_prober.go:29] interesting pod/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 18:56:17.844824 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.844794    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 18:56:17.859318 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.858273    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-maas-api-tls\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.880676 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.880514    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:17.881479 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.881456    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-maas-api-tls\") pod \"maas-api-e2e-mt-da58b769-c58747d4f-fc5b4\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:17.950053 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:17.949929    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:18.416179 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:18.407662    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:18.549471 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:18.549435    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"]
Oct 30 18:56:18.728398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:18.728324    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 18:56:18.731630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:18.731596    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 18:56:18.819125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:18.814371    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"
Oct 30 18:56:18.819632 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:18.819612    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"
Oct 30 18:56:18.843029 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:18.843003    4925 patch_prober.go:29] interesting pod/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 18:56:18.843142 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:18.843070    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 18:56:19.141057 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:19.141019    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4" event={"ID":"e573e700-5f11-4a17-9318-6d89ecbce82f","Type":"ContainerStarted","Data":"d13b42b76d5c38d5c6f262222ebac4bec4928c871ba0bdd727f1f1211e0ba3a1"}
Oct 30 18:56:19.389312 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:19.389123    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:19.847238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:19.847211    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:56:19.847738 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:19.847718    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:56:19.848356 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:19.848332    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:56:20.104305 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:20.103144    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:20.152898 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:20.152859    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4" event={"ID":"e573e700-5f11-4a17-9318-6d89ecbce82f","Type":"ContainerStarted","Data":"a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632"}
Oct 30 18:56:20.153144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:20.152947    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:20.170925 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:20.170876    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4" podStartSLOduration=3.627346307 podStartE2EDuration="4.170862687s" totalImagesPullingTime="543.51638ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:20.169360893 +0000 UTC m=+1201.885705971" watchObservedRunningTime="2026-10-30 18:56:20.170862687 +0000 UTC m=+1201.887207762"
Oct 30 18:56:20.790180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:20.780478    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:21.775747 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:21.775710    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:22.483495 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:22.483411    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:23.579127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:23.576346    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:23.900694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:23.900572    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:24.215357 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:24.209951    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"]
Oct 30 18:56:24.215357 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:24.210177    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8" podUID="9379191b-9d56-408d-9b33-215e889c7713" containerName="maas-api" containerID="cri-o://d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810" gracePeriod=30
Oct 30 18:56:24.306943 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:24.306889    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:24.484170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:24.484136    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:24.614118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:24.610297    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:56:24.697787 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:24.674549    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:24.971573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:24.954609    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-metrics-certs\") pod \"9379191b-9d56-408d-9b33-215e889c7713\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") "
Oct 30 18:56:24.971573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:24.954831    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9379191b-9d56-408d-9b33-215e889c7713-kube-api-access-b9rwt\" (UniqueName: \"kubernetes.io/projected/9379191b-9d56-408d-9b33-215e889c7713-kube-api-access-b9rwt\") pod \"9379191b-9d56-408d-9b33-215e889c7713\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") "
Oct 30 18:56:24.971573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:24.954972    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9379191b-9d56-408d-9b33-215e889c7713-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9379191b-9d56-408d-9b33-215e889c7713-openshift-service-ca\") pod \"9379191b-9d56-408d-9b33-215e889c7713\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") "
Oct 30 18:56:24.971573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:24.955195    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-maas-api-tls\") pod \"9379191b-9d56-408d-9b33-215e889c7713\" (UID: \"9379191b-9d56-408d-9b33-215e889c7713\") "
Oct 30 18:56:24.971573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:24.959596    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9379191b-9d56-408d-9b33-215e889c7713-openshift-service-ca" pod "9379191b-9d56-408d-9b33-215e889c7713" (UID: "9379191b-9d56-408d-9b33-215e889c7713"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:25.025920 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.020589    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-metrics-certs" pod "9379191b-9d56-408d-9b33-215e889c7713" (UID: "9379191b-9d56-408d-9b33-215e889c7713"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:56:25.025920 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.022940    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-maas-api-tls" pod "9379191b-9d56-408d-9b33-215e889c7713" (UID: "9379191b-9d56-408d-9b33-215e889c7713"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:56:25.025920 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.023251    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9379191b-9d56-408d-9b33-215e889c7713-kube-api-access-b9rwt" pod "9379191b-9d56-408d-9b33-215e889c7713" (UID: "9379191b-9d56-408d-9b33-215e889c7713"). InnerVolumeSpecName "kube-api-access-b9rwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:25.059291 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.059254    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:25.059291 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.059283    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9379191b-9d56-408d-9b33-215e889c7713-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:25.059507 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.059298    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9rwt\" (UniqueName: \"kubernetes.io/projected/9379191b-9d56-408d-9b33-215e889c7713-kube-api-access-b9rwt\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:25.059507 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.059313    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9379191b-9d56-408d-9b33-215e889c7713-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:25.091621 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.091571    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:25.221506 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.221470    4925 generic.go:356] "Generic (PLEG): container finished" podID="9379191b-9d56-408d-9b33-215e889c7713" containerID="d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810" exitCode=0
Oct 30 18:56:25.221717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.221510    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8" event={"ID":"9379191b-9d56-408d-9b33-215e889c7713","Type":"ContainerDied","Data":"d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810"}
Oct 30 18:56:25.221717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.221532    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"
Oct 30 18:56:25.221717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.221592    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8" event={"ID":"9379191b-9d56-408d-9b33-215e889c7713","Type":"ContainerDied","Data":"111efd2a68e161961f8a90a8d36ac52eee2f5967d85ca212b8db24bf5ad4a3b5"}
Oct 30 18:56:25.221717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.221626    4925 scope.go:122] "RemoveContainer" containerID="d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810"
Oct 30 18:56:25.243302 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.243286    4925 scope.go:122] "RemoveContainer" containerID="d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810"
Oct 30 18:56:25.243593 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:25.243572    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810\": container with ID starting with d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810 not found: ID does not exist" containerID="d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810"
Oct 30 18:56:25.243666 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.243604    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810"} err="failed to get container status \"d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810\": rpc error: code = NotFound desc = could not find container \"d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810\": container with ID starting with d7a71cc11b51b001ea060b93de95931f25f3e8574b1d6f993fb012fb8a9e9810 not found: ID does not exist"
Oct 30 18:56:25.253975 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.253953    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"]
Oct 30 18:56:25.259894 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.259866    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dc0f87-5d4cd9c954-k76n8"]
Oct 30 18:56:25.908133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:25.907736    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:26.424811 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:26.424782    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9379191b-9d56-408d-9b33-215e889c7713" path="/var/lib/kubelet/pods/9379191b-9d56-408d-9b33-215e889c7713/volumes"
Oct 30 18:56:26.678185 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:26.673139    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:27.186173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:27.186145    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:27.567008 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:27.566927    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:27.757315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:27.757282    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:28.365119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:28.364545    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:28.653577 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:28.653444    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:28.847411 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:28.847361    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:29.072531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.072496    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:29.160355 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.160315    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"
Oct 30 18:56:29.160708 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.160687    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"
Oct 30 18:56:29.767692 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.767657    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:29.797769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.797738    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"]
Oct 30 18:56:29.798901 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.798880    4925 cpu_manager.go:414] "containerMap: removing container" podUID="9379191b-9d56-408d-9b33-215e889c7713" containerName="maas-api"
Oct 30 18:56:29.798948 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.798905    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9379191b-9d56-408d-9b33-215e889c7713" containerName="maas-api"
Oct 30 18:56:29.799069 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.799056    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="9379191b-9d56-408d-9b33-215e889c7713" containerName="maas-api"
Oct 30 18:56:29.804777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.804757    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"
Oct 30 18:56:29.812253 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.812219    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-24fceb-dockercfg-drlp8\""
Oct 30 18:56:29.814442 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.814410    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"]
Oct 30 18:56:29.852535 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.852510    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"]
Oct 30 18:56:29.858113 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.858075    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"
Oct 30 18:56:29.878322 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.878295    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"]
Oct 30 18:56:29.881078 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.880326    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-24fceb\""
Oct 30 18:56:29.902759 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.902724    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmtpt\" (UniqueName: \"kubernetes.io/projected/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-kube-api-access-tmtpt\") pod \"payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj\" (UID: \"fce94553-c12c-40a8-9d4a-8ab6d34daa6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"
Oct 30 18:56:29.903017 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.902995    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj\" (UID: \"fce94553-c12c-40a8-9d4a-8ab6d34daa6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"
Oct 30 18:56:29.997863 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:29.997822    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:30.018336 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.018198    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj\" (UID: \"fce94553-c12c-40a8-9d4a-8ab6d34daa6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"
Oct 30 18:56:30.020869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.020837    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d78e4b6-f593-4a27-bca3-9dff6c1da055-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf\" (UID: \"7d78e4b6-f593-4a27-bca3-9dff6c1da055\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"
Oct 30 18:56:30.052312 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.052273    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgttl\" (UniqueName: \"kubernetes.io/projected/7d78e4b6-f593-4a27-bca3-9dff6c1da055-kube-api-access-lgttl\") pod \"payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf\" (UID: \"7d78e4b6-f593-4a27-bca3-9dff6c1da055\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"
Oct 30 18:56:30.052862 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.052834    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmtpt\" (UniqueName: \"kubernetes.io/projected/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-kube-api-access-tmtpt\") pod \"payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj\" (UID: \"fce94553-c12c-40a8-9d4a-8ab6d34daa6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"
Oct 30 18:56:30.066544 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.066274    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj\" (UID: \"fce94553-c12c-40a8-9d4a-8ab6d34daa6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"
Oct 30 18:56:30.167792 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.167755    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d78e4b6-f593-4a27-bca3-9dff6c1da055-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf\" (UID: \"7d78e4b6-f593-4a27-bca3-9dff6c1da055\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"
Oct 30 18:56:30.192253 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.192226    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgttl\" (UniqueName: \"kubernetes.io/projected/7d78e4b6-f593-4a27-bca3-9dff6c1da055-kube-api-access-lgttl\") pod \"payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf\" (UID: \"7d78e4b6-f593-4a27-bca3-9dff6c1da055\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"
Oct 30 18:56:30.193019 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.183318    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmtpt\" (UniqueName: \"kubernetes.io/projected/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-kube-api-access-tmtpt\") pod \"payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj\" (UID: \"fce94553-c12c-40a8-9d4a-8ab6d34daa6d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"
Oct 30 18:56:30.198451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.198375    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d78e4b6-f593-4a27-bca3-9dff6c1da055-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf\" (UID: \"7d78e4b6-f593-4a27-bca3-9dff6c1da055\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"
Oct 30 18:56:30.222941 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.222915    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgttl\" (UniqueName: \"kubernetes.io/projected/7d78e4b6-f593-4a27-bca3-9dff6c1da055-kube-api-access-lgttl\") pod \"payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf\" (UID: \"7d78e4b6-f593-4a27-bca3-9dff6c1da055\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"
Oct 30 18:56:30.229873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.229848    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"
Oct 30 18:56:30.242336 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.242291    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:30.357618 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.357191    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"]
Oct 30 18:56:30.362899 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.362872    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.380405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.378838    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-e7e190-openshift-default-dockercfg-mrd5q\""
Oct 30 18:56:30.386795 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.386767    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"]
Oct 30 18:56:30.397031 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.396997    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zc95\" (UniqueName: \"kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-kube-api-access-7zc95\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.397879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.397077    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-socket\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.397879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.397227    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17503242-eada-4ef5-b87a-8c766b072494-istiod-ca-cert\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.397879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.397279    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-envoy\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.397879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.397323    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-credential-socket\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.397879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.397456    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-data\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.397879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.397520    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-istio-token\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.397879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.397548    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-certs\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.397879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.397696    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17503242-eada-4ef5-b87a-8c766b072494-istio-podinfo\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.425490 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.425468    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"
Oct 30 18:56:30.491159 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.491080    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:30.499033 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.499003    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17503242-eada-4ef5-b87a-8c766b072494-istiod-ca-cert\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.499194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.499076    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-envoy\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.499194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.499167    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-credential-socket\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.499311 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.499262    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-data\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.499383 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.499318    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-istio-token\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.499383 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.499342    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-certs\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.499508 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.499489    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17503242-eada-4ef5-b87a-8c766b072494-istio-podinfo\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.499573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.499557    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zc95\" (UniqueName: \"kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-kube-api-access-7zc95\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.499639 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.499610    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-socket\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.499786 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.499765    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17503242-eada-4ef5-b87a-8c766b072494-istiod-ca-cert\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.500022 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.499999    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-socket\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.500311 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.500287    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-certs\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.500842 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.500820    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-credential-socket\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.501090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.501053    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-data\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.503050 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.502984    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-envoy\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.509161 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.506636    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17503242-eada-4ef5-b87a-8c766b072494-istio-podinfo\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.515324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.515297    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-istio-token\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.518759 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.518696    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zc95\" (UniqueName: \"kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-kube-api-access-7zc95\") pod \"e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") " pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.536227 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:56:30.536199    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d78e4b6_f593_4a27_bca3_9dff6c1da055.slice/crio-c15c9ee3d9d79136c3c27123cc40449fdaa839d9226087e5366be4c6e1a7d8a0 WatchSource:0}: Error finding container c15c9ee3d9d79136c3c27123cc40449fdaa839d9226087e5366be4c6e1a7d8a0: Status 404 returned error can't find the container with id c15c9ee3d9d79136c3c27123cc40449fdaa839d9226087e5366be4c6e1a7d8a0
Oct 30 18:56:30.542004 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.541961    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"]
Oct 30 18:56:30.723953 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.723910    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:30.744735 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.744701    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:30.800210 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:30.800066    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"]
Oct 30 18:56:31.065625 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.065581    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:31.318176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.318152    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"]
Oct 30 18:56:31.320261 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:56:31.320227    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17503242_eada_4ef5_b87a_8c766b072494.slice/crio-23b4423a77c9ac42d533fedb8dcb890d60bb2f29eb7cbeb67867816c8ebc5602 WatchSource:0}: Error finding container 23b4423a77c9ac42d533fedb8dcb890d60bb2f29eb7cbeb67867816c8ebc5602: Status 404 returned error can't find the container with id 23b4423a77c9ac42d533fedb8dcb890d60bb2f29eb7cbeb67867816c8ebc5602
Oct 30 18:56:31.322881 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.322847    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:56:31.322980 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.322916    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:56:31.322980 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.322944    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:56:31.392180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.392147    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf" event={"ID":"7d78e4b6-f593-4a27-bca3-9dff6c1da055","Type":"ContainerStarted","Data":"581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4"}
Oct 30 18:56:31.392300 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.392212    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf" event={"ID":"7d78e4b6-f593-4a27-bca3-9dff6c1da055","Type":"ContainerStarted","Data":"c15c9ee3d9d79136c3c27123cc40449fdaa839d9226087e5366be4c6e1a7d8a0"}
Oct 30 18:56:31.392300 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.392271    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"
Oct 30 18:56:31.394574 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.394552    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" event={"ID":"17503242-eada-4ef5-b87a-8c766b072494","Type":"ContainerStarted","Data":"23b4423a77c9ac42d533fedb8dcb890d60bb2f29eb7cbeb67867816c8ebc5602"}
Oct 30 18:56:31.396281 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.396262    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj" event={"ID":"fce94553-c12c-40a8-9d4a-8ab6d34daa6d","Type":"ContainerStarted","Data":"f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312"}
Oct 30 18:56:31.396364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.396302    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj" event={"ID":"fce94553-c12c-40a8-9d4a-8ab6d34daa6d","Type":"ContainerStarted","Data":"7a3f027f2d813040a99bfd6ff1a77f5174eb038698b683d20a88b889df52f0ea"}
Oct 30 18:56:31.396364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.396346    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"
Oct 30 18:56:31.419667 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.419602    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf" podStartSLOduration=2.4195826719999998 podStartE2EDuration="2.419582672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:31.414143616 +0000 UTC m=+1213.130488689" watchObservedRunningTime="2026-10-30 18:56:31.419582672 +0000 UTC m=+1213.135927750"
Oct 30 18:56:31.439456 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:31.439401    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj" podStartSLOduration=2.439383474 podStartE2EDuration="2.439383474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:31.434952601 +0000 UTC m=+1213.151297680" watchObservedRunningTime="2026-10-30 18:56:31.439383474 +0000 UTC m=+1213.155728552"
Oct 30 18:56:32.162200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:32.159933    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:32.355683 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:32.355643    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:32.408580 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:32.408540    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" event={"ID":"17503242-eada-4ef5-b87a-8c766b072494","Type":"ContainerStarted","Data":"da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192"}
Oct 30 18:56:32.434488 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:32.434375    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" podStartSLOduration=2.434358301 podStartE2EDuration="2.434358301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:32.430990303 +0000 UTC m=+1214.147335381" watchObservedRunningTime="2026-10-30 18:56:32.434358301 +0000 UTC m=+1214.150703378"
Oct 30 18:56:32.515467 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:32.515437    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"]
Oct 30 18:56:32.515708 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:32.515681    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy" containerID="cri-o://c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0" gracePeriod=30
Oct 30 18:56:32.670128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:32.669978    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:32.751176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:32.750824    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:32.780484 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:32.780444    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:32.876378 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:32.876327    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:33.082456 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:33.082368    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:33.415772 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:33.415731    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:33.416403 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:33.416387    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:56:33.731183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:33.730175    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:34.216628 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.216593    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:34.284120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.283395    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"]
Oct 30 18:56:34.471330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.471245    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"
Oct 30 18:56:34.520140 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.515776    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-12a6e1-dockercfg-gsw65\""
Oct 30 18:56:34.540119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.521253    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-12a6e1\""
Oct 30 18:56:34.540119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.524534    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"]
Oct 30 18:56:34.540119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.535619    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:34.558315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.552341    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86fcq\" (UniqueName: \"kubernetes.io/projected/4b33bf67-2556-4670-9fc9-fbe0e5e52877-kube-api-access-86fcq\") pod \"payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p\" (UID: \"4b33bf67-2556-4670-9fc9-fbe0e5e52877\") " pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"
Oct 30 18:56:34.558315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.553346    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b33bf67-2556-4670-9fc9-fbe0e5e52877-plugins-config-volume\") pod \"payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p\" (UID: \"4b33bf67-2556-4670-9fc9-fbe0e5e52877\") " pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"
Oct 30 18:56:34.655157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.655082    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b33bf67-2556-4670-9fc9-fbe0e5e52877-plugins-config-volume\") pod \"payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p\" (UID: \"4b33bf67-2556-4670-9fc9-fbe0e5e52877\") " pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"
Oct 30 18:56:34.655324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.655201    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86fcq\" (UniqueName: \"kubernetes.io/projected/4b33bf67-2556-4670-9fc9-fbe0e5e52877-kube-api-access-86fcq\") pod \"payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p\" (UID: \"4b33bf67-2556-4670-9fc9-fbe0e5e52877\") " pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"
Oct 30 18:56:34.655757 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.655735    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b33bf67-2556-4670-9fc9-fbe0e5e52877-plugins-config-volume\") pod \"payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p\" (UID: \"4b33bf67-2556-4670-9fc9-fbe0e5e52877\") " pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"
Oct 30 18:56:34.667141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.667117    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86fcq\" (UniqueName: \"kubernetes.io/projected/4b33bf67-2556-4670-9fc9-fbe0e5e52877-kube-api-access-86fcq\") pod \"payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p\" (UID: \"4b33bf67-2556-4670-9fc9-fbe0e5e52877\") " pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"
Oct 30 18:56:34.849765 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.849688    4925 patch_prober.go:29] interesting pod/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 18:56:34.849765 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.849750    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:34.864231 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.864196    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7496-be5069"]
Oct 30 18:56:34.877458 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.874365    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7496-be5069"]
Oct 30 18:56:34.877458 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.874465    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-be5069"
Oct 30 18:56:34.884410 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.884381    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-c64f8\""
Oct 30 18:56:34.906947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.906917    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"
Oct 30 18:56:34.965495 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:34.960025    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2z2w\" (UniqueName: \"kubernetes.io/projected/7a483469-57bf-49b9-af74-8259f4369b5a-kube-api-access-x2z2w\") pod \"test-curl-7496-be5069\" (UID: \"7a483469-57bf-49b9-af74-8259f4369b5a\") " pod="opendatahub/test-curl-7496-be5069"
Oct 30 18:56:35.061235 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:35.061204    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2z2w\" (UniqueName: \"kubernetes.io/projected/7a483469-57bf-49b9-af74-8259f4369b5a-kube-api-access-x2z2w\") pod \"test-curl-7496-be5069\" (UID: \"7a483469-57bf-49b9-af74-8259f4369b5a\") " pod="opendatahub/test-curl-7496-be5069"
Oct 30 18:56:35.083218 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:35.083182    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2z2w\" (UniqueName: \"kubernetes.io/projected/7a483469-57bf-49b9-af74-8259f4369b5a-kube-api-access-x2z2w\") pod \"test-curl-7496-be5069\" (UID: \"7a483469-57bf-49b9-af74-8259f4369b5a\") " pod="opendatahub/test-curl-7496-be5069"
Oct 30 18:56:35.141127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:35.141035    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:35.196931 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:35.196908    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"]
Oct 30 18:56:35.198266 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:35.198237    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-be5069"
Oct 30 18:56:35.362426 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:35.343910    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:35.497689 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:35.497653    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p" event={"ID":"4b33bf67-2556-4670-9fc9-fbe0e5e52877","Type":"ContainerStarted","Data":"73e62dceefa099f2403207ab4564b46b6be7d72c86d2f8de520de583cfbf99bf"}
Oct 30 18:56:35.742805 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:35.742778    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7496-be5069"]
Oct 30 18:56:36.364232 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:36.364198    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:36.528120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:36.528055    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p" event={"ID":"4b33bf67-2556-4670-9fc9-fbe0e5e52877","Type":"ContainerStarted","Data":"7a52a93636461e83f68b1df616e634c682682e2eca06bf08a51c088817a193c9"}
Oct 30 18:56:36.558154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:36.528770    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"
Oct 30 18:56:36.558154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:36.531659    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7496-be5069" event={"ID":"7a483469-57bf-49b9-af74-8259f4369b5a","Type":"ContainerStarted","Data":"fe145b91fee9654334e6a35893bc5520a93aa73581cf8a8477e4b743496cbef2"}
Oct 30 18:56:36.558154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:36.531699    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7496-be5069" event={"ID":"7a483469-57bf-49b9-af74-8259f4369b5a","Type":"ContainerStarted","Data":"293b832d146621f93d9eb724529ae5c61d30afabe2c7b7f66a96d0614d6ac05b"}
Oct 30 18:56:36.649211 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:36.644377    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p" podStartSLOduration=2.644357209 podStartE2EDuration="2.644357209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:36.546480794 +0000 UTC m=+1218.262825871" watchObservedRunningTime="2026-10-30 18:56:36.644357209 +0000 UTC m=+1218.360702286"
Oct 30 18:56:36.649211 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:36.648133    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7496-be5069" podStartSLOduration=2.648116934 podStartE2EDuration="2.648116934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:36.642184816 +0000 UTC m=+1218.358529893" watchObservedRunningTime="2026-10-30 18:56:36.648116934 +0000 UTC m=+1218.364462012"
Oct 30 18:56:36.663124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:36.655872    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:37.418682 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.418635    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7496-be5069"]
Oct 30 18:56:37.422871 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.422841    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7496-be5069"]
Oct 30 18:56:37.558191 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.553878    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:37.859966 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.859942    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:56:37.935050 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.935009    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-token\" (UniqueName: \"kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-token\") pod \"086f5d84-6529-4e59-bbc9-ec0141cd16de\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") "
Oct 30 18:56:37.935230 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.935144    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-certs\") pod \"086f5d84-6529-4e59-bbc9-ec0141cd16de\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") "
Oct 30 18:56:37.935300 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.935227    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-podinfo\") pod \"086f5d84-6529-4e59-bbc9-ec0141cd16de\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") "
Oct 30 18:56:37.935300 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.935279    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-envoy\") pod \"086f5d84-6529-4e59-bbc9-ec0141cd16de\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") "
Oct 30 18:56:37.935427 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.935327    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-socket\") pod \"086f5d84-6529-4e59-bbc9-ec0141cd16de\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") "
Oct 30 18:56:37.935513 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.935487    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-certs" pod "086f5d84-6529-4e59-bbc9-ec0141cd16de" (UID: "086f5d84-6529-4e59-bbc9-ec0141cd16de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:37.935780 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.935758    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-socket" pod "086f5d84-6529-4e59-bbc9-ec0141cd16de" (UID: "086f5d84-6529-4e59-bbc9-ec0141cd16de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:37.935884 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.935861    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/086f5d84-6529-4e59-bbc9-ec0141cd16de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086f5d84-6529-4e59-bbc9-ec0141cd16de-istiod-ca-cert\") pod \"086f5d84-6529-4e59-bbc9-ec0141cd16de\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") "
Oct 30 18:56:37.935952 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.935910    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-kube-api-access-h2jvf\" (UniqueName: \"kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-kube-api-access-h2jvf\") pod \"086f5d84-6529-4e59-bbc9-ec0141cd16de\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") "
Oct 30 18:56:37.936020 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.935957    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-credential-socket\") pod \"086f5d84-6529-4e59-bbc9-ec0141cd16de\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") "
Oct 30 18:56:37.936073 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.936033    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-data\") pod \"086f5d84-6529-4e59-bbc9-ec0141cd16de\" (UID: \"086f5d84-6529-4e59-bbc9-ec0141cd16de\") "
Oct 30 18:56:37.936482 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.936211    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/086f5d84-6529-4e59-bbc9-ec0141cd16de-istiod-ca-cert" pod "086f5d84-6529-4e59-bbc9-ec0141cd16de" (UID: "086f5d84-6529-4e59-bbc9-ec0141cd16de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:37.936482 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.936452    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-credential-socket" pod "086f5d84-6529-4e59-bbc9-ec0141cd16de" (UID: "086f5d84-6529-4e59-bbc9-ec0141cd16de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:37.936683 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.936522    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-data" pod "086f5d84-6529-4e59-bbc9-ec0141cd16de" (UID: "086f5d84-6529-4e59-bbc9-ec0141cd16de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:37.937081 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.937058    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086f5d84-6529-4e59-bbc9-ec0141cd16de-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:37.937190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.937087    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:37.937190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.937117    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:37.937190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.937131    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:37.937190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.937144    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:37.938355 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.938331    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-token" pod "086f5d84-6529-4e59-bbc9-ec0141cd16de" (UID: "086f5d84-6529-4e59-bbc9-ec0141cd16de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:37.938778 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.938753    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-kube-api-access-h2jvf" pod "086f5d84-6529-4e59-bbc9-ec0141cd16de" (UID: "086f5d84-6529-4e59-bbc9-ec0141cd16de"). InnerVolumeSpecName "kube-api-access-h2jvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:37.939250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.939230    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-envoy" pod "086f5d84-6529-4e59-bbc9-ec0141cd16de" (UID: "086f5d84-6529-4e59-bbc9-ec0141cd16de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:37.939488 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:37.939463    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-podinfo" pod "086f5d84-6529-4e59-bbc9-ec0141cd16de" (UID: "086f5d84-6529-4e59-bbc9-ec0141cd16de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:56:38.038424 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.038360    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:38.038424 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.038386    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:38.038424 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.038396    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2jvf\" (UniqueName: \"kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-kube-api-access-h2jvf\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:38.038424 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.038406    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086f5d84-6529-4e59-bbc9-ec0141cd16de-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:38.578166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.573688    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:38.614972 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.614263    4925 generic.go:356] "Generic (PLEG): container finished" podID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerID="c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0" exitCode=0
Oct 30 18:56:38.614972 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.614452    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7496-be5069" podUID="7a483469-57bf-49b9-af74-8259f4369b5a" containerName="test-curl-7496-be5069" containerID="cri-o://fe145b91fee9654334e6a35893bc5520a93aa73581cf8a8477e4b743496cbef2" gracePeriod=2
Oct 30 18:56:38.614972 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.614737    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"
Oct 30 18:56:38.615394 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.615373    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" event={"ID":"086f5d84-6529-4e59-bbc9-ec0141cd16de","Type":"ContainerDied","Data":"c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0"}
Oct 30 18:56:38.615574 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.615560    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh" event={"ID":"086f5d84-6529-4e59-bbc9-ec0141cd16de","Type":"ContainerDied","Data":"4a456917d94633ac29452e0f1859463f466aa9dfe15ed77b3cc0b54d8ee8e80e"}
Oct 30 18:56:38.615655 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.615647    4925 scope.go:122] "RemoveContainer" containerID="c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0"
Oct 30 18:56:38.628531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.628509    4925 scope.go:122] "RemoveContainer" containerID="852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442"
Oct 30 18:56:38.639033 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.639012    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"]
Oct 30 18:56:38.639394 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.639236    4925 scope.go:122] "RemoveContainer" containerID="c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0"
Oct 30 18:56:38.639661 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:38.639639    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0\": container with ID starting with c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0 not found: ID does not exist" containerID="c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0"
Oct 30 18:56:38.639764 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.639667    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0"} err="failed to get container status \"c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0\": rpc error: code = NotFound desc = could not find container \"c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0\": container with ID starting with c50786e2d0fa5d74c4bce94c14fb8c015d1bde09623429cd3f08a06756542fb0 not found: ID does not exist"
Oct 30 18:56:38.639764 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.639686    4925 scope.go:122] "RemoveContainer" containerID="852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442"
Oct 30 18:56:38.640012 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:38.639987    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442\": container with ID starting with 852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442 not found: ID does not exist" containerID="852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442"
Oct 30 18:56:38.640138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.640019    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442"} err="failed to get container status \"852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442\": rpc error: code = NotFound desc = could not find container \"852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442\": container with ID starting with 852585ed21e0aed9b7d19ea030ae4a3f9d25d9ca68b9dddca3ae6383a3429442 not found: ID does not exist"
Oct 30 18:56:38.643385 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:38.643366    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-dc0f87-openshift-default-66c7fb4749-7c7vh"]
Oct 30 18:56:39.608128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:39.607995    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:40.426368 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.426325    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" path="/var/lib/kubelet/pods/086f5d84-6529-4e59-bbc9-ec0141cd16de/volumes"
Oct 30 18:56:40.850974 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.850950    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-be5069"
Oct 30 18:56:40.930292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.930110    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a483469-57bf-49b9-af74-8259f4369b5a-kube-api-access-x2z2w\" (UniqueName: \"kubernetes.io/projected/7a483469-57bf-49b9-af74-8259f4369b5a-kube-api-access-x2z2w\") pod \"7a483469-57bf-49b9-af74-8259f4369b5a\" (UID: \"7a483469-57bf-49b9-af74-8259f4369b5a\") "
Oct 30 18:56:40.938321 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.938286    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a483469-57bf-49b9-af74-8259f4369b5a-kube-api-access-x2z2w" pod "7a483469-57bf-49b9-af74-8259f4369b5a" (UID: "7a483469-57bf-49b9-af74-8259f4369b5a"). InnerVolumeSpecName "kube-api-access-x2z2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:40.954290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.950896    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"]
Oct 30 18:56:40.954290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.952719    4925 cpu_manager.go:414] "containerMap: removing container" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy"
Oct 30 18:56:40.954290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.952739    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy"
Oct 30 18:56:40.954290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.952773    4925 cpu_manager.go:414] "containerMap: removing container" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy"
Oct 30 18:56:40.954290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.952781    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy"
Oct 30 18:56:40.954290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.952807    4925 cpu_manager.go:414] "containerMap: removing container" podUID="7a483469-57bf-49b9-af74-8259f4369b5a" containerName="test-curl-7496-be5069"
Oct 30 18:56:40.954290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.952815    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a483469-57bf-49b9-af74-8259f4369b5a" containerName="test-curl-7496-be5069"
Oct 30 18:56:40.954290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.953200    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy"
Oct 30 18:56:40.954290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.953226    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="086f5d84-6529-4e59-bbc9-ec0141cd16de" containerName="istio-proxy"
Oct 30 18:56:40.954290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.953244    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a483469-57bf-49b9-af74-8259f4369b5a" containerName="test-curl-7496-be5069"
Oct 30 18:56:40.963173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.963138    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:40.964767 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.964742    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"]
Oct 30 18:56:40.969363 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:40.969341    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-24fceb-serving-cert\""
Oct 30 18:56:41.033987 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.033927    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-metrics-certs\") pod \"maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.034123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.033986    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.034123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.034053    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6dbfed-a952-4c82-be12-9d477b34978c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.034239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.034225    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zdwg\" (UniqueName: \"kubernetes.io/projected/ea6dbfed-a952-4c82-be12-9d477b34978c-kube-api-access-9zdwg\") pod \"maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.034370 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.034358    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2z2w\" (UniqueName: \"kubernetes.io/projected/7a483469-57bf-49b9-af74-8259f4369b5a-kube-api-access-x2z2w\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:41.088948 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.088925    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:41.136348 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.136319    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zdwg\" (UniqueName: \"kubernetes.io/projected/ea6dbfed-a952-4c82-be12-9d477b34978c-kube-api-access-9zdwg\") pod \"maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.136534 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.136517    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-metrics-certs\") pod \"maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.136617 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.136603    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.136655 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.136637    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6dbfed-a952-4c82-be12-9d477b34978c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.137622 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.137593    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6dbfed-a952-4c82-be12-9d477b34978c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.139434 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.139407    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-metrics-certs\") pod \"maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.139562 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.139458    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.197124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.195878    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zdwg\" (UniqueName: \"kubernetes.io/projected/ea6dbfed-a952-4c82-be12-9d477b34978c-kube-api-access-9zdwg\") pod \"maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.274137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.267690    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:41.288391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.288299    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:41.623186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.619575    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"]
Oct 30 18:56:41.683122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.679480    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s" event={"ID":"ea6dbfed-a952-4c82-be12-9d477b34978c","Type":"ContainerStarted","Data":"e3d44c8d58885605ab3d56c5f353281c567104079dcbeb10f1af18d5f94b095e"}
Oct 30 18:56:41.709047 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.705737    4925 generic.go:356] "Generic (PLEG): container finished" podID="7a483469-57bf-49b9-af74-8259f4369b5a" containerID="fe145b91fee9654334e6a35893bc5520a93aa73581cf8a8477e4b743496cbef2" exitCode=137
Oct 30 18:56:41.709047 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.706027    4925 scope.go:122] "RemoveContainer" containerID="fe145b91fee9654334e6a35893bc5520a93aa73581cf8a8477e4b743496cbef2"
Oct 30 18:56:41.709047 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.706271    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-be5069"
Oct 30 18:56:41.729115 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.726049    4925 scope.go:122] "RemoveContainer" containerID="fe145b91fee9654334e6a35893bc5520a93aa73581cf8a8477e4b743496cbef2"
Oct 30 18:56:41.729115 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:41.726377    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe145b91fee9654334e6a35893bc5520a93aa73581cf8a8477e4b743496cbef2\": container with ID starting with fe145b91fee9654334e6a35893bc5520a93aa73581cf8a8477e4b743496cbef2 not found: ID does not exist" containerID="fe145b91fee9654334e6a35893bc5520a93aa73581cf8a8477e4b743496cbef2"
Oct 30 18:56:41.729115 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.726403    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe145b91fee9654334e6a35893bc5520a93aa73581cf8a8477e4b743496cbef2"} err="failed to get container status \"fe145b91fee9654334e6a35893bc5520a93aa73581cf8a8477e4b743496cbef2\": rpc error: code = NotFound desc = could not find container \"fe145b91fee9654334e6a35893bc5520a93aa73581cf8a8477e4b743496cbef2\": container with ID starting with fe145b91fee9654334e6a35893bc5520a93aa73581cf8a8477e4b743496cbef2 not found: ID does not exist"
Oct 30 18:56:41.729115 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:41.727892    4925 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7496-be5069\" is forbidden: User \"system:node:ip-10-0-134-58.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-58.us-east-2.compute.internal' and this object" podUID="7a483469-57bf-49b9-af74-8259f4369b5a" pod="opendatahub/test-curl-7496-be5069"
Oct 30 18:56:42.000117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:41.994589    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:42.411562 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:42.411523    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"
Oct 30 18:56:42.411724 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:42.411589    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"
Oct 30 18:56:42.424130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:42.424083    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a483469-57bf-49b9-af74-8259f4369b5a" path="/var/lib/kubelet/pods/7a483469-57bf-49b9-af74-8259f4369b5a/volumes"
Oct 30 18:56:42.433071 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:42.433039    4925 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7496-be5069\" is forbidden: User \"system:node:ip-10-0-134-58.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-58.us-east-2.compute.internal' and this object" podUID="7a483469-57bf-49b9-af74-8259f4369b5a" pod="opendatahub/test-curl-7496-be5069"
Oct 30 18:56:42.689175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:42.688706    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:42.741221 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:42.740486    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s" event={"ID":"ea6dbfed-a952-4c82-be12-9d477b34978c","Type":"ContainerStarted","Data":"b0fdda1d8b5ad93666f982b527a5c468106d036360a011c193859a5bc3da7b18"}
Oct 30 18:56:42.741221 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:42.740547    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:42.760604 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:42.760559    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s" podStartSLOduration=2.314847065 podStartE2EDuration="2.760545635s" totalImagesPullingTime="445.69857ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:42.758605737 +0000 UTC m=+1224.474950818" watchObservedRunningTime="2026-10-30 18:56:42.760545635 +0000 UTC m=+1224.476890711"
Oct 30 18:56:43.126469 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:43.126389    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"]
Oct 30 18:56:43.140898 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:43.140867    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"]
Oct 30 18:56:43.281118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:43.281055    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:43.309766 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:43.309705    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:56:43.476121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:43.473468    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:43.669121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:43.665658    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:44.390137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:44.388448    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:45.102397 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:45.102348    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:45.278730 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:45.278670    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:45.525120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:45.521852    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:45.874189 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:45.874081    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:46.826701 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:46.826665    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"]
Oct 30 18:56:46.838625 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:46.833643    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:46.838625 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:46.838033    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-e7e190-serving-cert\""
Oct 30 18:56:46.841722 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:46.841266    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"]
Oct 30 18:56:46.925517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:46.925487    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-metrics-certs\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:46.925716 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:46.925694    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db5672e5-8c89-43f8-bb92-82917a137bbd-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:46.925794 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:46.925741    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:46.925858 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:46.925843    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvr96\" (UniqueName: \"kubernetes.io/projected/db5672e5-8c89-43f8-bb92-82917a137bbd-kube-api-access-bvr96\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:46.966157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:46.959389    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:47.040118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.036114    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvr96\" (UniqueName: \"kubernetes.io/projected/db5672e5-8c89-43f8-bb92-82917a137bbd-kube-api-access-bvr96\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:47.040118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.036242    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-metrics-certs\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:47.040118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.036487    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db5672e5-8c89-43f8-bb92-82917a137bbd-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:47.040118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.036552    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:47.040118 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:47.036740    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-serving-cert: secret "maas-api-e2e-ipp-b-e7e190-serving-cert" not found
Oct 30 18:56:47.040118 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:47.036805    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-maas-api-tls podName:db5672e5-8c89-43f8-bb92-82917a137bbd nodeName:}" failed. No retries permitted until 2026-10-30 18:56:47.536784567 +0000 UTC m=+1229.253129640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-maas-api-tls") pod "maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7" (UID: "db5672e5-8c89-43f8-bb92-82917a137bbd") : secret "maas-api-e2e-ipp-b-e7e190-serving-cert" not found
Oct 30 18:56:47.040118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.037809    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db5672e5-8c89-43f8-bb92-82917a137bbd-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:47.068119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.051075    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-metrics-certs\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:47.068119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.053011    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvr96\" (UniqueName: \"kubernetes.io/projected/db5672e5-8c89-43f8-bb92-82917a137bbd-kube-api-access-bvr96\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:47.135569 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.135474    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:47.326351 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.326319    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:47.548564 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.548525    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:47.582140 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.579029    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:47.618182 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.610293    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"
Oct 30 18:56:47.755630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.755600    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:47.912360 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.901144    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:47.912360 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.912044    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"]
Oct 30 18:56:47.914057 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:47.914030    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"]
Oct 30 18:56:48.053126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.052987    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"]
Oct 30 18:56:48.054002 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:56:48.053972    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5672e5_8c89_43f8_bb92_82917a137bbd.slice/crio-80854d619a980e6e1ac4df62046c0cba81442ca27b1b2f6ce56e8a293ae96009 WatchSource:0}: Error finding container 80854d619a980e6e1ac4df62046c0cba81442ca27b1b2f6ce56e8a293ae96009: Status 404 returned error can't find the container with id 80854d619a980e6e1ac4df62046c0cba81442ca27b1b2f6ce56e8a293ae96009
Oct 30 18:56:48.089502 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.089432    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:48.127576 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.127541    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp" podUID="5bfe6c2a-58d2-4a50-bedf-ef345ada9266" containerName="payload-processing" containerID="cri-o://8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd" gracePeriod=25
Oct 30 18:56:48.141533 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.141502    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6" podUID="af7e9f27-86db-4f00-a966-412d1c25eb2f" containerName="payload-pre-processing" containerID="cri-o://7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220" gracePeriod=25
Oct 30 18:56:48.282598 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.282566    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:48.666676 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.666641    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"
Oct 30 18:56:48.720292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.720266    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"
Oct 30 18:56:48.763607 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.763575    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 18:56:48.781614 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.781577    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af7e9f27-86db-4f00-a966-412d1c25eb2f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af7e9f27-86db-4f00-a966-412d1c25eb2f-plugins-config-volume\") pod \"af7e9f27-86db-4f00-a966-412d1c25eb2f\" (UID: \"af7e9f27-86db-4f00-a966-412d1c25eb2f\") "
Oct 30 18:56:48.781924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.781824    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af7e9f27-86db-4f00-a966-412d1c25eb2f-kube-api-access-mrxwm\" (UniqueName: \"kubernetes.io/projected/af7e9f27-86db-4f00-a966-412d1c25eb2f-kube-api-access-mrxwm\") pod \"af7e9f27-86db-4f00-a966-412d1c25eb2f\" (UID: \"af7e9f27-86db-4f00-a966-412d1c25eb2f\") "
Oct 30 18:56:48.782044 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.781981    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-plugins-config-volume\") pod \"5bfe6c2a-58d2-4a50-bedf-ef345ada9266\" (UID: \"5bfe6c2a-58d2-4a50-bedf-ef345ada9266\") "
Oct 30 18:56:48.782143 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.782052    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-kube-api-access-bpqhw\" (UniqueName: \"kubernetes.io/projected/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-kube-api-access-bpqhw\") pod \"5bfe6c2a-58d2-4a50-bedf-ef345ada9266\" (UID: \"5bfe6c2a-58d2-4a50-bedf-ef345ada9266\") "
Oct 30 18:56:48.782143 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.782126    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af7e9f27-86db-4f00-a966-412d1c25eb2f-plugins-config-volume" pod "af7e9f27-86db-4f00-a966-412d1c25eb2f" (UID: "af7e9f27-86db-4f00-a966-412d1c25eb2f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:48.782717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.782675    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-plugins-config-volume" pod "5bfe6c2a-58d2-4a50-bedf-ef345ada9266" (UID: "5bfe6c2a-58d2-4a50-bedf-ef345ada9266"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:48.782814 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.782791    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af7e9f27-86db-4f00-a966-412d1c25eb2f-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:48.785833 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.785799    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-kube-api-access-bpqhw" pod "5bfe6c2a-58d2-4a50-bedf-ef345ada9266" (UID: "5bfe6c2a-58d2-4a50-bedf-ef345ada9266"). InnerVolumeSpecName "kube-api-access-bpqhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:48.786614 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.786588    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af7e9f27-86db-4f00-a966-412d1c25eb2f-kube-api-access-mrxwm" pod "af7e9f27-86db-4f00-a966-412d1c25eb2f" (UID: "af7e9f27-86db-4f00-a966-412d1c25eb2f"). InnerVolumeSpecName "kube-api-access-mrxwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:48.810251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.810213    4925 generic.go:356] "Generic (PLEG): container finished" podID="5bfe6c2a-58d2-4a50-bedf-ef345ada9266" containerID="8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd" exitCode=0
Oct 30 18:56:48.810400 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.810297    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp" event={"ID":"5bfe6c2a-58d2-4a50-bedf-ef345ada9266","Type":"ContainerDied","Data":"8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd"}
Oct 30 18:56:48.810400 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.810342    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp" event={"ID":"5bfe6c2a-58d2-4a50-bedf-ef345ada9266","Type":"ContainerDied","Data":"99a2efc62a0f84b69db43f0574f373455453362692d665e13c9eecf3320cc55d"}
Oct 30 18:56:48.810400 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.810367    4925 scope.go:122] "RemoveContainer" containerID="8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd"
Oct 30 18:56:48.810569 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.810495    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"
Oct 30 18:56:48.818278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.814626    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7" event={"ID":"db5672e5-8c89-43f8-bb92-82917a137bbd","Type":"ContainerStarted","Data":"be4deb66846b987a9ed4b4c58951f13bb05392d97b14aad8b9dfe0f9902d9468"}
Oct 30 18:56:48.818278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.814676    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7" event={"ID":"db5672e5-8c89-43f8-bb92-82917a137bbd","Type":"ContainerStarted","Data":"80854d619a980e6e1ac4df62046c0cba81442ca27b1b2f6ce56e8a293ae96009"}
Oct 30 18:56:48.818278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.814745    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:48.818278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.816388    4925 generic.go:356] "Generic (PLEG): container finished" podID="af7e9f27-86db-4f00-a966-412d1c25eb2f" containerID="7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220" exitCode=0
Oct 30 18:56:48.818278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.816435    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"
Oct 30 18:56:48.818278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.816468    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6" event={"ID":"af7e9f27-86db-4f00-a966-412d1c25eb2f","Type":"ContainerDied","Data":"7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220"}
Oct 30 18:56:48.818278 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.816516    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6" event={"ID":"af7e9f27-86db-4f00-a966-412d1c25eb2f","Type":"ContainerDied","Data":"c0b646e63104893023ce593e51a619365b0e38d396c7dafd0104e7af09fe5fbb"}
Oct 30 18:56:48.832620 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.832469    4925 scope.go:122] "RemoveContainer" containerID="8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd"
Oct 30 18:56:48.837403 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:48.837304    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd\": container with ID starting with 8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd not found: ID does not exist" containerID="8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd"
Oct 30 18:56:48.837403 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.837337    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd"} err="failed to get container status \"8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd\": rpc error: code = NotFound desc = could not find container \"8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd\": container with ID starting with 8a00b0719a4dad72950bfd9d9bc196fd64c684fbcf4b8719b71b3fa96bddaabd not found: ID does not exist"
Oct 30 18:56:48.837403 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.837356    4925 scope.go:122] "RemoveContainer" containerID="7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220"
Oct 30 18:56:48.841676 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.841624    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7" podStartSLOduration=2.347408632 podStartE2EDuration="2.841608145s" totalImagesPullingTime="494.199513ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:48.839395251 +0000 UTC m=+1230.555740326" watchObservedRunningTime="2026-10-30 18:56:48.841608145 +0000 UTC m=+1230.557953222"
Oct 30 18:56:48.853480 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.853115    4925 scope.go:122] "RemoveContainer" containerID="7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220"
Oct 30 18:56:48.853886 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:48.853822    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220\": container with ID starting with 7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220 not found: ID does not exist" containerID="7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220"
Oct 30 18:56:48.853886 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.853857    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220"} err="failed to get container status \"7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220\": rpc error: code = NotFound desc = could not find container \"7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220\": container with ID starting with 7c65ffd84e74b2c6f82f8954d30a51d0d99b7b626dfa7e7856acb6df4b69f220 not found: ID does not exist"
Oct 30 18:56:48.855652 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.855629    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"]
Oct 30 18:56:48.859865 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.859841    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-9982ad-b9cb8c9cd-t92mp"]
Oct 30 18:56:48.872050 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.872030    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"]
Oct 30 18:56:48.876284 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.876264    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9982ad-56b9dc5dd7-b94s6"]
Oct 30 18:56:48.884842 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.884817    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:48.884953 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.884889    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpqhw\" (UniqueName: \"kubernetes.io/projected/5bfe6c2a-58d2-4a50-bedf-ef345ada9266-kube-api-access-bpqhw\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:48.884953 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.884908    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrxwm\" (UniqueName: \"kubernetes.io/projected/af7e9f27-86db-4f00-a966-412d1c25eb2f-kube-api-access-mrxwm\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:48.887571 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:48.887547    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:49.795654 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:49.795621    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:50.466128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:50.465461    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5bfe6c2a-58d2-4a50-bedf-ef345ada9266" path="/var/lib/kubelet/pods/5bfe6c2a-58d2-4a50-bedf-ef345ada9266/volumes"
Oct 30 18:56:50.487124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:50.482307    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af7e9f27-86db-4f00-a966-412d1c25eb2f" path="/var/lib/kubelet/pods/af7e9f27-86db-4f00-a966-412d1c25eb2f/volumes"
Oct 30 18:56:50.487124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:50.485085    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:51.193705 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:51.193677    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:51.766923 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:51.766889    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:52.139169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.136880    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"]
Oct 30 18:56:52.139169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.137564    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4" podUID="e573e700-5f11-4a17-9318-6d89ecbce82f" containerName="maas-api" containerID="cri-o://a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632" gracePeriod=30
Oct 30 18:56:52.172791 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.172755    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4" podUID="e573e700-5f11-4a17-9318-6d89ecbce82f" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.114:8443/health\": dial tcp 10.132.0.114:8443: connect: connection refused"
Oct 30 18:56:52.284588 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.284551    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:52.726063 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.726034    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:52.744352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.744331    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:52.759198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.759163    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-maas-api-tls\") pod \"e573e700-5f11-4a17-9318-6d89ecbce82f\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") "
Oct 30 18:56:52.759343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.759298    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-metrics-certs\") pod \"e573e700-5f11-4a17-9318-6d89ecbce82f\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") "
Oct 30 18:56:52.759408 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.759362    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e573e700-5f11-4a17-9318-6d89ecbce82f-kube-api-access-tvfnz\" (UniqueName: \"kubernetes.io/projected/e573e700-5f11-4a17-9318-6d89ecbce82f-kube-api-access-tvfnz\") pod \"e573e700-5f11-4a17-9318-6d89ecbce82f\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") "
Oct 30 18:56:52.759532 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.759517    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e573e700-5f11-4a17-9318-6d89ecbce82f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e573e700-5f11-4a17-9318-6d89ecbce82f-openshift-service-ca\") pod \"e573e700-5f11-4a17-9318-6d89ecbce82f\" (UID: \"e573e700-5f11-4a17-9318-6d89ecbce82f\") "
Oct 30 18:56:52.759912 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.759878    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e573e700-5f11-4a17-9318-6d89ecbce82f-openshift-service-ca" pod "e573e700-5f11-4a17-9318-6d89ecbce82f" (UID: "e573e700-5f11-4a17-9318-6d89ecbce82f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:52.760225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.760204    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e573e700-5f11-4a17-9318-6d89ecbce82f-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:52.762243 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.761736    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e573e700-5f11-4a17-9318-6d89ecbce82f-kube-api-access-tvfnz" pod "e573e700-5f11-4a17-9318-6d89ecbce82f" (UID: "e573e700-5f11-4a17-9318-6d89ecbce82f"). InnerVolumeSpecName "kube-api-access-tvfnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:52.763623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.763584    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-metrics-certs" pod "e573e700-5f11-4a17-9318-6d89ecbce82f" (UID: "e573e700-5f11-4a17-9318-6d89ecbce82f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:56:52.769752 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.769711    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-maas-api-tls" pod "e573e700-5f11-4a17-9318-6d89ecbce82f" (UID: "e573e700-5f11-4a17-9318-6d89ecbce82f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:56:52.853559 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.853523    4925 generic.go:356] "Generic (PLEG): container finished" podID="e573e700-5f11-4a17-9318-6d89ecbce82f" containerID="a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632" exitCode=0
Oct 30 18:56:52.853709 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.853573    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"
Oct 30 18:56:52.853709 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.853616    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4" event={"ID":"e573e700-5f11-4a17-9318-6d89ecbce82f","Type":"ContainerDied","Data":"a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632"}
Oct 30 18:56:52.853709 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.853676    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4" event={"ID":"e573e700-5f11-4a17-9318-6d89ecbce82f","Type":"ContainerDied","Data":"d13b42b76d5c38d5c6f262222ebac4bec4928c871ba0bdd727f1f1211e0ba3a1"}
Oct 30 18:56:52.853709 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.853705    4925 scope.go:122] "RemoveContainer" containerID="a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632"
Oct 30 18:56:52.861306 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.861273    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:52.861306 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.861307    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e573e700-5f11-4a17-9318-6d89ecbce82f-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:52.861505 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.861324    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvfnz\" (UniqueName: \"kubernetes.io/projected/e573e700-5f11-4a17-9318-6d89ecbce82f-kube-api-access-tvfnz\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:52.865466 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.865443    4925 scope.go:122] "RemoveContainer" containerID="a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632"
Oct 30 18:56:52.865763 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:52.865742    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632\": container with ID starting with a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632 not found: ID does not exist" containerID="a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632"
Oct 30 18:56:52.865849 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.865770    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632"} err="failed to get container status \"a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632\": rpc error: code = NotFound desc = could not find container \"a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632\": container with ID starting with a070c01a8c7539f7d3dfa88e87c10baab5a42fc7ce4ce5c24d571810cea04632 not found: ID does not exist"
Oct 30 18:56:52.911066 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.911028    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:52.915614 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.915539    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb" podUID="97ca9fe2-b463-4ade-be86-b4ffa7a34cef" containerName="payload-pre-processing" containerID="cri-o://b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e" gracePeriod=25
Oct 30 18:56:52.933564 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.920639    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75" podUID="595201e2-381f-488d-a987-cb741ac1254b" containerName="payload-processing" containerID="cri-o://d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00" gracePeriod=25
Oct 30 18:56:52.968334 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.964395    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"]
Oct 30 18:56:52.979565 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:52.979530    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-da58b769-c58747d4f-fc5b4"]
Oct 30 18:56:53.289938 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.289906    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"
Oct 30 18:56:53.330389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.330336    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:53.341129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.341085    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"
Oct 30 18:56:53.375173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.375143    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/595201e2-381f-488d-a987-cb741ac1254b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/595201e2-381f-488d-a987-cb741ac1254b-plugins-config-volume\") pod \"595201e2-381f-488d-a987-cb741ac1254b\" (UID: \"595201e2-381f-488d-a987-cb741ac1254b\") "
Oct 30 18:56:53.375543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.375409    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-kube-api-access-f7s9d\" (UniqueName: \"kubernetes.io/projected/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-kube-api-access-f7s9d\") pod \"97ca9fe2-b463-4ade-be86-b4ffa7a34cef\" (UID: \"97ca9fe2-b463-4ade-be86-b4ffa7a34cef\") "
Oct 30 18:56:53.375543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.375528    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/595201e2-381f-488d-a987-cb741ac1254b-plugins-config-volume" pod "595201e2-381f-488d-a987-cb741ac1254b" (UID: "595201e2-381f-488d-a987-cb741ac1254b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:53.375723 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.375608    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/595201e2-381f-488d-a987-cb741ac1254b-kube-api-access-jjbp5\" (UniqueName: \"kubernetes.io/projected/595201e2-381f-488d-a987-cb741ac1254b-kube-api-access-jjbp5\") pod \"595201e2-381f-488d-a987-cb741ac1254b\" (UID: \"595201e2-381f-488d-a987-cb741ac1254b\") "
Oct 30 18:56:53.375723 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.375686    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-plugins-config-volume\") pod \"97ca9fe2-b463-4ade-be86-b4ffa7a34cef\" (UID: \"97ca9fe2-b463-4ade-be86-b4ffa7a34cef\") "
Oct 30 18:56:53.376279 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.376250    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/595201e2-381f-488d-a987-cb741ac1254b-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:53.376640 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.376614    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-plugins-config-volume" pod "97ca9fe2-b463-4ade-be86-b4ffa7a34cef" (UID: "97ca9fe2-b463-4ade-be86-b4ffa7a34cef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:53.378348 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.378319    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/595201e2-381f-488d-a987-cb741ac1254b-kube-api-access-jjbp5" pod "595201e2-381f-488d-a987-cb741ac1254b" (UID: "595201e2-381f-488d-a987-cb741ac1254b"). InnerVolumeSpecName "kube-api-access-jjbp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:53.378563 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.378533    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-kube-api-access-f7s9d" pod "97ca9fe2-b463-4ade-be86-b4ffa7a34cef" (UID: "97ca9fe2-b463-4ade-be86-b4ffa7a34cef"). InnerVolumeSpecName "kube-api-access-f7s9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:53.444965 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.444933    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"]
Oct 30 18:56:53.452244 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.452215    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"]
Oct 30 18:56:53.479433 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.479393    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7s9d\" (UniqueName: \"kubernetes.io/projected/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-kube-api-access-f7s9d\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:53.479433 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.479429    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjbp5\" (UniqueName: \"kubernetes.io/projected/595201e2-381f-488d-a987-cb741ac1254b-kube-api-access-jjbp5\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:53.479691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.479446    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97ca9fe2-b463-4ade-be86-b4ffa7a34cef-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:53.620262 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.620141    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:53.827141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.815494    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:53.917739 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.911938    4925 generic.go:356] "Generic (PLEG): container finished" podID="97ca9fe2-b463-4ade-be86-b4ffa7a34cef" containerID="b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e" exitCode=0
Oct 30 18:56:53.917739 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.912443    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb" event={"ID":"97ca9fe2-b463-4ade-be86-b4ffa7a34cef","Type":"ContainerDied","Data":"b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e"}
Oct 30 18:56:53.917739 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.912623    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb" event={"ID":"97ca9fe2-b463-4ade-be86-b4ffa7a34cef","Type":"ContainerDied","Data":"64f74f32683676398be829d6fc887ac163cd80a416e052274dc36952193a3302"}
Oct 30 18:56:53.917739 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.912691    4925 scope.go:122] "RemoveContainer" containerID="b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e"
Oct 30 18:56:53.917739 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.916752    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"
Oct 30 18:56:53.922585 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.921210    4925 generic.go:356] "Generic (PLEG): container finished" podID="595201e2-381f-488d-a987-cb741ac1254b" containerID="d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00" exitCode=0
Oct 30 18:56:53.922585 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.921317    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75" event={"ID":"595201e2-381f-488d-a987-cb741ac1254b","Type":"ContainerDied","Data":"d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00"}
Oct 30 18:56:53.922585 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.921361    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75" event={"ID":"595201e2-381f-488d-a987-cb741ac1254b","Type":"ContainerDied","Data":"cf347ce5cc142ae829668528236302988a1d38294666d469f099a72e6253c04d"}
Oct 30 18:56:53.922585 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:53.921431    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"
Oct 30 18:56:54.035740 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.035710    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:54.064234 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.064199    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"]
Oct 30 18:56:54.064442 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.064418    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-da58b769-7ddb646744-dvj75"]
Oct 30 18:56:54.078293 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.078268    4925 scope.go:122] "RemoveContainer" containerID="b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e"
Oct 30 18:56:54.079063 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:54.079022    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e\": container with ID starting with b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e not found: ID does not exist" containerID="b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e"
Oct 30 18:56:54.079196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.079059    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e"} err="failed to get container status \"b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e\": rpc error: code = NotFound desc = could not find container \"b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e\": container with ID starting with b5e7b27c4d1bd85117b2b88ab929ff3c7e501737f13274233854d8330c4bf46e not found: ID does not exist"
Oct 30 18:56:54.079196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.079085    4925 scope.go:122] "RemoveContainer" containerID="d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00"
Oct 30 18:56:54.080480 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.080461    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"]
Oct 30 18:56:54.083843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.083820    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-da58b769-786c599f47-kbbwb"]
Oct 30 18:56:54.090164 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.090146    4925 scope.go:122] "RemoveContainer" containerID="d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00"
Oct 30 18:56:54.090462 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:54.090443    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00\": container with ID starting with d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00 not found: ID does not exist" containerID="d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00"
Oct 30 18:56:54.090517 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.090471    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00"} err="failed to get container status \"d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00\": rpc error: code = NotFound desc = could not find container \"d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00\": container with ID starting with d965d0679f9a13ac65d082de414d35a65124cc9c01fdfd0b8fda31243edcec00 not found: ID does not exist"
Oct 30 18:56:54.424728 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.424688    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="595201e2-381f-488d-a987-cb741ac1254b" path="/var/lib/kubelet/pods/595201e2-381f-488d-a987-cb741ac1254b/volumes"
Oct 30 18:56:54.425298 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.425246    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97ca9fe2-b463-4ade-be86-b4ffa7a34cef" path="/var/lib/kubelet/pods/97ca9fe2-b463-4ade-be86-b4ffa7a34cef/volumes"
Oct 30 18:56:54.425788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.425773    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e573e700-5f11-4a17-9318-6d89ecbce82f" path="/var/lib/kubelet/pods/e573e700-5f11-4a17-9318-6d89ecbce82f/volumes"
Oct 30 18:56:54.548123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.546282    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:54.834503 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:54.834421    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:56:55.212759 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:55.212719    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:56.137156 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:56.136613    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:56.995138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:56.995083    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:57.945348 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:57.945319    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:58.135287 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:58.135231    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:58.242777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:58.242697    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"]
Oct 30 18:56:58.409141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:58.409085    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:58.445539 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:58.445500    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf" podUID="7d78e4b6-f593-4a27-bca3-9dff6c1da055" containerName="payload-processing" containerID="cri-o://581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4" gracePeriod=25
Oct 30 18:56:58.453023 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:58.452989    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj" podUID="fce94553-c12c-40a8-9d4a-8ab6d34daa6d" containerName="payload-pre-processing" containerID="cri-o://f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312" gracePeriod=25
Oct 30 18:56:58.661118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:58.660645    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:58.907556 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:58.907526    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"
Oct 30 18:56:58.941622 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:58.940251    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"
Oct 30 18:56:59.044400 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.044363    4925 generic.go:356] "Generic (PLEG): container finished" podID="fce94553-c12c-40a8-9d4a-8ab6d34daa6d" containerID="f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312" exitCode=0
Oct 30 18:56:59.044897 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.044462    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"
Oct 30 18:56:59.044897 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.044510    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj" event={"ID":"fce94553-c12c-40a8-9d4a-8ab6d34daa6d","Type":"ContainerDied","Data":"f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312"}
Oct 30 18:56:59.044897 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.044573    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj" event={"ID":"fce94553-c12c-40a8-9d4a-8ab6d34daa6d","Type":"ContainerDied","Data":"7a3f027f2d813040a99bfd6ff1a77f5174eb038698b683d20a88b889df52f0ea"}
Oct 30 18:56:59.044897 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.044608    4925 scope.go:122] "RemoveContainer" containerID="f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312"
Oct 30 18:56:59.045904 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.045879    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d78e4b6-f593-4a27-bca3-9dff6c1da055-kube-api-access-lgttl\" (UniqueName: \"kubernetes.io/projected/7d78e4b6-f593-4a27-bca3-9dff6c1da055-kube-api-access-lgttl\") pod \"7d78e4b6-f593-4a27-bca3-9dff6c1da055\" (UID: \"7d78e4b6-f593-4a27-bca3-9dff6c1da055\") "
Oct 30 18:56:59.046047 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.045937    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-plugins-config-volume\") pod \"fce94553-c12c-40a8-9d4a-8ab6d34daa6d\" (UID: \"fce94553-c12c-40a8-9d4a-8ab6d34daa6d\") "
Oct 30 18:56:59.046135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.046054    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d78e4b6-f593-4a27-bca3-9dff6c1da055-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d78e4b6-f593-4a27-bca3-9dff6c1da055-plugins-config-volume\") pod \"7d78e4b6-f593-4a27-bca3-9dff6c1da055\" (UID: \"7d78e4b6-f593-4a27-bca3-9dff6c1da055\") "
Oct 30 18:56:59.046207 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.046188    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-kube-api-access-tmtpt\" (UniqueName: \"kubernetes.io/projected/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-kube-api-access-tmtpt\") pod \"fce94553-c12c-40a8-9d4a-8ab6d34daa6d\" (UID: \"fce94553-c12c-40a8-9d4a-8ab6d34daa6d\") "
Oct 30 18:56:59.046954 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.046668    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d78e4b6-f593-4a27-bca3-9dff6c1da055-plugins-config-volume" pod "7d78e4b6-f593-4a27-bca3-9dff6c1da055" (UID: "7d78e4b6-f593-4a27-bca3-9dff6c1da055"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:59.047120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.047077    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d78e4b6-f593-4a27-bca3-9dff6c1da055-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:59.047520 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.047491    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-plugins-config-volume" pod "fce94553-c12c-40a8-9d4a-8ab6d34daa6d" (UID: "fce94553-c12c-40a8-9d4a-8ab6d34daa6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:59.049078 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.048407    4925 generic.go:356] "Generic (PLEG): container finished" podID="7d78e4b6-f593-4a27-bca3-9dff6c1da055" containerID="581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4" exitCode=0
Oct 30 18:56:59.049078 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.048514    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf" event={"ID":"7d78e4b6-f593-4a27-bca3-9dff6c1da055","Type":"ContainerDied","Data":"581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4"}
Oct 30 18:56:59.049078 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.048560    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf" event={"ID":"7d78e4b6-f593-4a27-bca3-9dff6c1da055","Type":"ContainerDied","Data":"c15c9ee3d9d79136c3c27123cc40449fdaa839d9226087e5366be4c6e1a7d8a0"}
Oct 30 18:56:59.049078 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.048634    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"
Oct 30 18:56:59.049360 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.049261    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d78e4b6-f593-4a27-bca3-9dff6c1da055-kube-api-access-lgttl" pod "7d78e4b6-f593-4a27-bca3-9dff6c1da055" (UID: "7d78e4b6-f593-4a27-bca3-9dff6c1da055"). InnerVolumeSpecName "kube-api-access-lgttl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:59.049868 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.049847    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-kube-api-access-tmtpt" pod "fce94553-c12c-40a8-9d4a-8ab6d34daa6d" (UID: "fce94553-c12c-40a8-9d4a-8ab6d34daa6d"). InnerVolumeSpecName "kube-api-access-tmtpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:59.058935 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.058913    4925 scope.go:122] "RemoveContainer" containerID="f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312"
Oct 30 18:56:59.059309 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:59.059285    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312\": container with ID starting with f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312 not found: ID does not exist" containerID="f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312"
Oct 30 18:56:59.059414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.059317    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312"} err="failed to get container status \"f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312\": rpc error: code = NotFound desc = could not find container \"f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312\": container with ID starting with f007e5dbcfdb7bf2c337d3fcc4637b0e95314a8d6e6361a66253878d0b977312 not found: ID does not exist"
Oct 30 18:56:59.059414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.059334    4925 scope.go:122] "RemoveContainer" containerID="581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4"
Oct 30 18:56:59.071807 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.071781    4925 scope.go:122] "RemoveContainer" containerID="581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4"
Oct 30 18:56:59.072237 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:56:59.072166    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4\": container with ID starting with 581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4 not found: ID does not exist" containerID="581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4"
Oct 30 18:56:59.072340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.072249    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4"} err="failed to get container status \"581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4\": rpc error: code = NotFound desc = could not find container \"581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4\": container with ID starting with 581a97ab3a44a019fa3fcf3b3dc86a17e43f369540338027cbf0dc3092c86fa4 not found: ID does not exist"
Oct 30 18:56:59.148687 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.148607    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgttl\" (UniqueName: \"kubernetes.io/projected/7d78e4b6-f593-4a27-bca3-9dff6c1da055-kube-api-access-lgttl\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:59.148687 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.148635    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:59.148687 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.148645    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmtpt\" (UniqueName: \"kubernetes.io/projected/fce94553-c12c-40a8-9d4a-8ab6d34daa6d-kube-api-access-tmtpt\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:56:59.369086 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.369060    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"]
Oct 30 18:56:59.375676 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.375648    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-5b8577b5f5-g62jj"]
Oct 30 18:56:59.389847 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.389809    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"]
Oct 30 18:56:59.389944 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.389858    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-24fceb-7784669d4b-8ztnf"]
Oct 30 18:56:59.774526 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:56:59.774441    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:00.096319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:00.096229    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:00.296060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:00.288222    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:00.477118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:00.476420    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d78e4b6-f593-4a27-bca3-9dff6c1da055" path="/var/lib/kubelet/pods/7d78e4b6-f593-4a27-bca3-9dff6c1da055/volumes"
Oct 30 18:57:00.477326 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:00.477158    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fce94553-c12c-40a8-9d4a-8ab6d34daa6d" path="/var/lib/kubelet/pods/fce94553-c12c-40a8-9d4a-8ab6d34daa6d/volumes"
Oct 30 18:57:00.497120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:00.487463    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:00.871173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:00.867827    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:01.547122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:01.543622    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:01.918066 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:01.918030    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"]
Oct 30 18:57:01.918369 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:01.918340    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw" podUID="c938f52d-8915-41a8-968d-253e99f1f490" containerName="maas-api" containerID="cri-o://bbdc83e6e370c3879c21097e10d698f4444448647409cfc99f6257e16ee205c4" gracePeriod=30
Oct 30 18:57:02.063558 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.059002    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:02.081455 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.081414    4925 generic.go:356] "Generic (PLEG): container finished" podID="c938f52d-8915-41a8-968d-253e99f1f490" containerID="bbdc83e6e370c3879c21097e10d698f4444448647409cfc99f6257e16ee205c4" exitCode=0
Oct 30 18:57:02.081647 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.081616    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw" event={"ID":"c938f52d-8915-41a8-968d-253e99f1f490","Type":"ContainerDied","Data":"bbdc83e6e370c3879c21097e10d698f4444448647409cfc99f6257e16ee205c4"}
Oct 30 18:57:02.274180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.270007    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:02.384834 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.383343    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:57:02.546211 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.546079    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-metrics-certs\") pod \"c938f52d-8915-41a8-968d-253e99f1f490\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") "
Oct 30 18:57:02.546211 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.546172    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-maas-api-tls\") pod \"c938f52d-8915-41a8-968d-253e99f1f490\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") "
Oct 30 18:57:02.546420 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.546273    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c938f52d-8915-41a8-968d-253e99f1f490-kube-api-access-8x4rh\" (UniqueName: \"kubernetes.io/projected/c938f52d-8915-41a8-968d-253e99f1f490-kube-api-access-8x4rh\") pod \"c938f52d-8915-41a8-968d-253e99f1f490\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") "
Oct 30 18:57:02.546420 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.546336    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c938f52d-8915-41a8-968d-253e99f1f490-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c938f52d-8915-41a8-968d-253e99f1f490-openshift-service-ca\") pod \"c938f52d-8915-41a8-968d-253e99f1f490\" (UID: \"c938f52d-8915-41a8-968d-253e99f1f490\") "
Oct 30 18:57:02.546810 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.546774    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c938f52d-8915-41a8-968d-253e99f1f490-openshift-service-ca" pod "c938f52d-8915-41a8-968d-253e99f1f490" (UID: "c938f52d-8915-41a8-968d-253e99f1f490"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:02.547373 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.547345    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c938f52d-8915-41a8-968d-253e99f1f490-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:02.548528 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.548500    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-maas-api-tls" pod "c938f52d-8915-41a8-968d-253e99f1f490" (UID: "c938f52d-8915-41a8-968d-253e99f1f490"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:57:02.548645 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.548628    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-metrics-certs" pod "c938f52d-8915-41a8-968d-253e99f1f490" (UID: "c938f52d-8915-41a8-968d-253e99f1f490"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:57:02.548740 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.548720    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c938f52d-8915-41a8-968d-253e99f1f490-kube-api-access-8x4rh" pod "c938f52d-8915-41a8-968d-253e99f1f490" (UID: "c938f52d-8915-41a8-968d-253e99f1f490"). InnerVolumeSpecName "kube-api-access-8x4rh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:02.650398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.649751    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x4rh\" (UniqueName: \"kubernetes.io/projected/c938f52d-8915-41a8-968d-253e99f1f490-kube-api-access-8x4rh\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:02.650398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.649795    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:02.650398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.649812    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c938f52d-8915-41a8-968d-253e99f1f490-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:02.664763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.664621    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:02.859028 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:02.858936    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:03.092128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.092060    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"
Oct 30 18:57:03.092128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.092076    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw" event={"ID":"c938f52d-8915-41a8-968d-253e99f1f490","Type":"ContainerDied","Data":"7981be0a735397bd8ad447d8a3a140c27402f5ab5a88037c08867c945eb45a3d"}
Oct 30 18:57:03.092362 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.092192    4925 scope.go:122] "RemoveContainer" containerID="bbdc83e6e370c3879c21097e10d698f4444448647409cfc99f6257e16ee205c4"
Oct 30 18:57:03.126438 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.126368    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"]
Oct 30 18:57:03.133475 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.133449    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9982ad-8557fb44b-nl5vw"]
Oct 30 18:57:03.261271 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261242    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"]
Oct 30 18:57:03.261899 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261885    4925 cpu_manager.go:414] "containerMap: removing container" podUID="595201e2-381f-488d-a987-cb741ac1254b" containerName="payload-processing"
Oct 30 18:57:03.261944 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261902    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="595201e2-381f-488d-a987-cb741ac1254b" containerName="payload-processing"
Oct 30 18:57:03.261944 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261916    4925 cpu_manager.go:414] "containerMap: removing container" podUID="5bfe6c2a-58d2-4a50-bedf-ef345ada9266" containerName="payload-processing"
Oct 30 18:57:03.261944 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261922    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bfe6c2a-58d2-4a50-bedf-ef345ada9266" containerName="payload-processing"
Oct 30 18:57:03.261944 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261937    4925 cpu_manager.go:414] "containerMap: removing container" podUID="e573e700-5f11-4a17-9318-6d89ecbce82f" containerName="maas-api"
Oct 30 18:57:03.261944 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261942    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e573e700-5f11-4a17-9318-6d89ecbce82f" containerName="maas-api"
Oct 30 18:57:03.262090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261958    4925 cpu_manager.go:414] "containerMap: removing container" podUID="fce94553-c12c-40a8-9d4a-8ab6d34daa6d" containerName="payload-pre-processing"
Oct 30 18:57:03.262090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261963    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fce94553-c12c-40a8-9d4a-8ab6d34daa6d" containerName="payload-pre-processing"
Oct 30 18:57:03.262090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261976    4925 cpu_manager.go:414] "containerMap: removing container" podUID="7d78e4b6-f593-4a27-bca3-9dff6c1da055" containerName="payload-processing"
Oct 30 18:57:03.262090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261980    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d78e4b6-f593-4a27-bca3-9dff6c1da055" containerName="payload-processing"
Oct 30 18:57:03.262090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261992    4925 cpu_manager.go:414] "containerMap: removing container" podUID="97ca9fe2-b463-4ade-be86-b4ffa7a34cef" containerName="payload-pre-processing"
Oct 30 18:57:03.262090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.261997    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97ca9fe2-b463-4ade-be86-b4ffa7a34cef" containerName="payload-pre-processing"
Oct 30 18:57:03.262090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.262004    4925 cpu_manager.go:414] "containerMap: removing container" podUID="af7e9f27-86db-4f00-a966-412d1c25eb2f" containerName="payload-pre-processing"
Oct 30 18:57:03.262090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.262009    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af7e9f27-86db-4f00-a966-412d1c25eb2f" containerName="payload-pre-processing"
Oct 30 18:57:03.262090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.262035    4925 cpu_manager.go:414] "containerMap: removing container" podUID="c938f52d-8915-41a8-968d-253e99f1f490" containerName="maas-api"
Oct 30 18:57:03.262090 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.262039    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c938f52d-8915-41a8-968d-253e99f1f490" containerName="maas-api"
Oct 30 18:57:03.262414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.262154    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="c938f52d-8915-41a8-968d-253e99f1f490" containerName="maas-api"
Oct 30 18:57:03.262414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.262165    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="595201e2-381f-488d-a987-cb741ac1254b" containerName="payload-processing"
Oct 30 18:57:03.262414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.262172    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="5bfe6c2a-58d2-4a50-bedf-ef345ada9266" containerName="payload-processing"
Oct 30 18:57:03.262414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.262178    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="e573e700-5f11-4a17-9318-6d89ecbce82f" containerName="maas-api"
Oct 30 18:57:03.262414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.262187    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="fce94553-c12c-40a8-9d4a-8ab6d34daa6d" containerName="payload-pre-processing"
Oct 30 18:57:03.262414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.262195    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d78e4b6-f593-4a27-bca3-9dff6c1da055" containerName="payload-processing"
Oct 30 18:57:03.262414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.262201    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="97ca9fe2-b463-4ade-be86-b4ffa7a34cef" containerName="payload-pre-processing"
Oct 30 18:57:03.262414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.262207    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="af7e9f27-86db-4f00-a966-412d1c25eb2f" containerName="payload-pre-processing"
Oct 30 18:57:03.267163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.267141    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"
Oct 30 18:57:03.270779 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.270742    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-24fceb\""
Oct 30 18:57:03.271026 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.271005    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-24fceb-dockercfg-6srcs\""
Oct 30 18:57:03.272612 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.272412    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:03.275307 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.275285    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"]
Oct 30 18:57:03.299955 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.299924    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"]
Oct 30 18:57:03.308384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.308345    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"
Oct 30 18:57:03.313153 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.313128    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"]
Oct 30 18:57:03.361976 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.361946    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7eea034-caf9-4b41-8cbb-77715993d9d1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6\" (UID: \"c7eea034-caf9-4b41-8cbb-77715993d9d1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"
Oct 30 18:57:03.362173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.362151    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8\" (UID: \"b234d7d5-b327-43a5-ac64-aa2a377ae3a6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"
Oct 30 18:57:03.362235 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.362199    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmg9q\" (UniqueName: \"kubernetes.io/projected/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-kube-api-access-dmg9q\") pod \"payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8\" (UID: \"b234d7d5-b327-43a5-ac64-aa2a377ae3a6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"
Oct 30 18:57:03.362303 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.362280    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfw4c\" (UniqueName: \"kubernetes.io/projected/c7eea034-caf9-4b41-8cbb-77715993d9d1-kube-api-access-bfw4c\") pod \"payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6\" (UID: \"c7eea034-caf9-4b41-8cbb-77715993d9d1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"
Oct 30 18:57:03.478123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.469083    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7eea034-caf9-4b41-8cbb-77715993d9d1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6\" (UID: \"c7eea034-caf9-4b41-8cbb-77715993d9d1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"
Oct 30 18:57:03.478123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.469244    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8\" (UID: \"b234d7d5-b327-43a5-ac64-aa2a377ae3a6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"
Oct 30 18:57:03.478123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.469286    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmg9q\" (UniqueName: \"kubernetes.io/projected/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-kube-api-access-dmg9q\") pod \"payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8\" (UID: \"b234d7d5-b327-43a5-ac64-aa2a377ae3a6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"
Oct 30 18:57:03.478123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.469315    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfw4c\" (UniqueName: \"kubernetes.io/projected/c7eea034-caf9-4b41-8cbb-77715993d9d1-kube-api-access-bfw4c\") pod \"payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6\" (UID: \"c7eea034-caf9-4b41-8cbb-77715993d9d1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"
Oct 30 18:57:03.478123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.472651    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7eea034-caf9-4b41-8cbb-77715993d9d1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6\" (UID: \"c7eea034-caf9-4b41-8cbb-77715993d9d1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"
Oct 30 18:57:03.478123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.472881    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8\" (UID: \"b234d7d5-b327-43a5-ac64-aa2a377ae3a6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"
Oct 30 18:57:03.478123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.475355    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:03.528117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.521725    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfw4c\" (UniqueName: \"kubernetes.io/projected/c7eea034-caf9-4b41-8cbb-77715993d9d1-kube-api-access-bfw4c\") pod \"payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6\" (UID: \"c7eea034-caf9-4b41-8cbb-77715993d9d1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"
Oct 30 18:57:03.570199 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.566932    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmg9q\" (UniqueName: \"kubernetes.io/projected/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-kube-api-access-dmg9q\") pod \"payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8\" (UID: \"b234d7d5-b327-43a5-ac64-aa2a377ae3a6\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"
Oct 30 18:57:03.622136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.607439    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"
Oct 30 18:57:03.645148 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.630492    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"
Oct 30 18:57:03.795340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:03.795286    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:04.000975 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:04.000138    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:04.035756 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:04.035717    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"]
Oct 30 18:57:04.036563 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:57:04.036531    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7eea034_caf9_4b41_8cbb_77715993d9d1.slice/crio-e365114b56c6635ecbb22af39db9e0e9f09467878b7de4f30b170b676258747a WatchSource:0}: Error finding container e365114b56c6635ecbb22af39db9e0e9f09467878b7de4f30b170b676258747a: Status 404 returned error can't find the container with id e365114b56c6635ecbb22af39db9e0e9f09467878b7de4f30b170b676258747a
Oct 30 18:57:04.045192 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:04.045172    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"]
Oct 30 18:57:04.050108 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:57:04.050068    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb234d7d5_b327_43a5_ac64_aa2a377ae3a6.slice/crio-bd67922efd6495ec64738b8bc5f77af122c3db9028375b18dfab3a48f6b5f04b WatchSource:0}: Error finding container bd67922efd6495ec64738b8bc5f77af122c3db9028375b18dfab3a48f6b5f04b: Status 404 returned error can't find the container with id bd67922efd6495ec64738b8bc5f77af122c3db9028375b18dfab3a48f6b5f04b
Oct 30 18:57:04.102183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:04.102152    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6" event={"ID":"c7eea034-caf9-4b41-8cbb-77715993d9d1","Type":"ContainerStarted","Data":"e365114b56c6635ecbb22af39db9e0e9f09467878b7de4f30b170b676258747a"}
Oct 30 18:57:04.103504 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:04.103478    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8" event={"ID":"b234d7d5-b327-43a5-ac64-aa2a377ae3a6","Type":"ContainerStarted","Data":"bd67922efd6495ec64738b8bc5f77af122c3db9028375b18dfab3a48f6b5f04b"}
Oct 30 18:57:04.173142 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:04.172830    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:04.491119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:04.488565    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c938f52d-8915-41a8-968d-253e99f1f490" path="/var/lib/kubelet/pods/c938f52d-8915-41a8-968d-253e99f1f490/volumes"
Oct 30 18:57:05.073338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:05.073306    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:05.123122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:05.117657    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6" event={"ID":"c7eea034-caf9-4b41-8cbb-77715993d9d1","Type":"ContainerStarted","Data":"3dc7ec9857008f76254a92e460508995b32ba6a0cd7b5c82a01e6bc27464c750"}
Oct 30 18:57:05.123122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:05.118709    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"
Oct 30 18:57:05.123122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:05.121000    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8" event={"ID":"b234d7d5-b327-43a5-ac64-aa2a377ae3a6","Type":"ContainerStarted","Data":"b7b7cda22a317f25ec4fcb5ab8c5fedb91771004f78c3e33e8e793d869a4f332"}
Oct 30 18:57:05.123122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:05.121809    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"
Oct 30 18:57:05.189126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:05.188989    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6" podStartSLOduration=2.188968081 podStartE2EDuration="2.188968081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:05.167512522 +0000 UTC m=+1246.883857600" watchObservedRunningTime="2026-10-30 18:57:05.188968081 +0000 UTC m=+1246.905313157"
Oct 30 18:57:05.208116 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:05.205436    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8" podStartSLOduration=2.205416608 podStartE2EDuration="2.205416608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:05.187506873 +0000 UTC m=+1246.903851951" watchObservedRunningTime="2026-10-30 18:57:05.205416608 +0000 UTC m=+1246.921761685"
Oct 30 18:57:06.288120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:06.287800    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:07.008128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:07.007255    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:07.834162 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:07.832863    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:08.808122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:08.804288    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:09.083170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.082625    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:09.330813 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.330780    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"]
Oct 30 18:57:09.335463 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.335391    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"]
Oct 30 18:57:09.343925 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.342935    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:09.508162 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.508132    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"]
Oct 30 18:57:09.518437 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.518412    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"
Oct 30 18:57:09.526411 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.526386    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-e7e190-dockercfg-c7tlb\""
Oct 30 18:57:09.526606 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.526593    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"]
Oct 30 18:57:09.526885 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.526871    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-e7e190\""
Oct 30 18:57:09.549551 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.549520    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:09.551625 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.551598    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"]
Oct 30 18:57:09.603909 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.603834    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"
Oct 30 18:57:09.604205 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.603866    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"]
Oct 30 18:57:09.651338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.651312    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26dvg\" (UniqueName: \"kubernetes.io/projected/eaa10c34-997b-4362-92f3-e29f168e1659-kube-api-access-26dvg\") pod \"payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk\" (UID: \"eaa10c34-997b-4362-92f3-e29f168e1659\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"
Oct 30 18:57:09.651591 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.651573    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaa10c34-997b-4362-92f3-e29f168e1659-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk\" (UID: \"eaa10c34-997b-4362-92f3-e29f168e1659\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"
Oct 30 18:57:09.755325 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.753165    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26dvg\" (UniqueName: \"kubernetes.io/projected/eaa10c34-997b-4362-92f3-e29f168e1659-kube-api-access-26dvg\") pod \"payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk\" (UID: \"eaa10c34-997b-4362-92f3-e29f168e1659\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"
Oct 30 18:57:09.755325 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.753281    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0876bd1c-4d76-4118-9a9a-92c43398d4a5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg\" (UID: \"0876bd1c-4d76-4118-9a9a-92c43398d4a5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"
Oct 30 18:57:09.755325 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.753332    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaa10c34-997b-4362-92f3-e29f168e1659-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk\" (UID: \"eaa10c34-997b-4362-92f3-e29f168e1659\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"
Oct 30 18:57:09.755325 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.753439    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnhw6\" (UniqueName: \"kubernetes.io/projected/0876bd1c-4d76-4118-9a9a-92c43398d4a5-kube-api-access-hnhw6\") pod \"payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg\" (UID: \"0876bd1c-4d76-4118-9a9a-92c43398d4a5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"
Oct 30 18:57:09.755325 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.754894    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaa10c34-997b-4362-92f3-e29f168e1659-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk\" (UID: \"eaa10c34-997b-4362-92f3-e29f168e1659\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"
Oct 30 18:57:09.881172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.861548    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0876bd1c-4d76-4118-9a9a-92c43398d4a5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg\" (UID: \"0876bd1c-4d76-4118-9a9a-92c43398d4a5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"
Oct 30 18:57:09.881172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.861657    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnhw6\" (UniqueName: \"kubernetes.io/projected/0876bd1c-4d76-4118-9a9a-92c43398d4a5-kube-api-access-hnhw6\") pod \"payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg\" (UID: \"0876bd1c-4d76-4118-9a9a-92c43398d4a5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"
Oct 30 18:57:09.916498 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.894507    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0876bd1c-4d76-4118-9a9a-92c43398d4a5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg\" (UID: \"0876bd1c-4d76-4118-9a9a-92c43398d4a5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"
Oct 30 18:57:09.947726 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:09.926337    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnhw6\" (UniqueName: \"kubernetes.io/projected/0876bd1c-4d76-4118-9a9a-92c43398d4a5-kube-api-access-hnhw6\") pod \"payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg\" (UID: \"0876bd1c-4d76-4118-9a9a-92c43398d4a5\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"
Oct 30 18:57:10.082120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:10.079472    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26dvg\" (UniqueName: \"kubernetes.io/projected/eaa10c34-997b-4362-92f3-e29f168e1659-kube-api-access-26dvg\") pod \"payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk\" (UID: \"eaa10c34-997b-4362-92f3-e29f168e1659\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"
Oct 30 18:57:10.148777 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:10.148692    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"
Oct 30 18:57:10.238260 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:10.238224    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"
Oct 30 18:57:10.519943 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:10.489216    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:10.715456 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:57:10.715323    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaa10c34_997b_4362_92f3_e29f168e1659.slice/crio-c4549d8ffe032c21acf177408d3cd5ad52b9b7ee0506c9039f98f879f11c5a68 WatchSource:0}: Error finding container c4549d8ffe032c21acf177408d3cd5ad52b9b7ee0506c9039f98f879f11c5a68: Status 404 returned error can't find the container with id c4549d8ffe032c21acf177408d3cd5ad52b9b7ee0506c9039f98f879f11c5a68
Oct 30 18:57:10.716591 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:10.716501    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"]
Oct 30 18:57:10.721240 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:10.721220    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"]
Oct 30 18:57:11.232468 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:11.232433    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk" event={"ID":"eaa10c34-997b-4362-92f3-e29f168e1659","Type":"ContainerStarted","Data":"a607dd7582907f9cebb76ee20ccf394190022865e4c940599da5ccf5e6779acd"}
Oct 30 18:57:11.232993 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:11.232488    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk" event={"ID":"eaa10c34-997b-4362-92f3-e29f168e1659","Type":"ContainerStarted","Data":"c4549d8ffe032c21acf177408d3cd5ad52b9b7ee0506c9039f98f879f11c5a68"}
Oct 30 18:57:11.232993 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:11.232521    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"
Oct 30 18:57:11.234254 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:11.234230    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg" event={"ID":"0876bd1c-4d76-4118-9a9a-92c43398d4a5","Type":"ContainerStarted","Data":"5dc82032fc407faaf5f7630baecfc49af458681a432fcaee4691b02ed4793f52"}
Oct 30 18:57:11.234372 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:11.234282    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg" event={"ID":"0876bd1c-4d76-4118-9a9a-92c43398d4a5","Type":"ContainerStarted","Data":"9237c8995f00831b81a6dfb38d5b3c867e824862d1c6feb765c6b345a35ee7ab"}
Oct 30 18:57:11.234372 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:11.234314    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"
Oct 30 18:57:11.250239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:11.249744    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk" podStartSLOduration=2.249729054 podStartE2EDuration="2.249729054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:11.248841937 +0000 UTC m=+1252.965187013" watchObservedRunningTime="2026-10-30 18:57:11.249729054 +0000 UTC m=+1252.966074134"
Oct 30 18:57:11.286124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:11.278998    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg" podStartSLOduration=2.278979913 podStartE2EDuration="2.278979913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:11.276489027 +0000 UTC m=+1252.992834102" watchObservedRunningTime="2026-10-30 18:57:11.278979913 +0000 UTC m=+1252.995324991"
Oct 30 18:57:11.511583 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:11.508791    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:12.286389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:12.285420    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:12.878158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:12.878124    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:13.269751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.266693    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp" podUID="89e0f65a-5731-4b10-ae2b-19b559064edb" containerName="main" containerID="cri-o://24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831" gracePeriod=45
Oct 30 18:57:13.334559 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:57:13.334520    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:57:13.662029 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.661941    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:13.732902 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.732534    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:57:13.807120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.807023    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89e0f65a-5731-4b10-ae2b-19b559064edb-tls-certs\" (UniqueName: \"kubernetes.io/secret/89e0f65a-5731-4b10-ae2b-19b559064edb-tls-certs\") pod \"89e0f65a-5731-4b10-ae2b-19b559064edb\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") "
Oct 30 18:57:13.807311 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.807148    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-dshm\") pod \"89e0f65a-5731-4b10-ae2b-19b559064edb\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") "
Oct 30 18:57:13.807311 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.807191    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-model-cache\") pod \"89e0f65a-5731-4b10-ae2b-19b559064edb\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") "
Oct 30 18:57:13.807311 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.807268    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89e0f65a-5731-4b10-ae2b-19b559064edb-kube-api-access-7qwbg\" (UniqueName: \"kubernetes.io/projected/89e0f65a-5731-4b10-ae2b-19b559064edb-kube-api-access-7qwbg\") pod \"89e0f65a-5731-4b10-ae2b-19b559064edb\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") "
Oct 30 18:57:13.807311 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.807297    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-home\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-home\") pod \"89e0f65a-5731-4b10-ae2b-19b559064edb\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") "
Oct 30 18:57:13.807556 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.807374    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-tmp-dir\") pod \"89e0f65a-5731-4b10-ae2b-19b559064edb\" (UID: \"89e0f65a-5731-4b10-ae2b-19b559064edb\") "
Oct 30 18:57:13.808810 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.808783    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-model-cache" pod "89e0f65a-5731-4b10-ae2b-19b559064edb" (UID: "89e0f65a-5731-4b10-ae2b-19b559064edb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:13.809080 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.809060    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-tmp-dir" pod "89e0f65a-5731-4b10-ae2b-19b559064edb" (UID: "89e0f65a-5731-4b10-ae2b-19b559064edb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:13.809599 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.809470    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-home" pod "89e0f65a-5731-4b10-ae2b-19b559064edb" (UID: "89e0f65a-5731-4b10-ae2b-19b559064edb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:13.811885 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.811860    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-dshm" pod "89e0f65a-5731-4b10-ae2b-19b559064edb" (UID: "89e0f65a-5731-4b10-ae2b-19b559064edb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:13.812625 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.812590    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89e0f65a-5731-4b10-ae2b-19b559064edb-tls-certs" pod "89e0f65a-5731-4b10-ae2b-19b559064edb" (UID: "89e0f65a-5731-4b10-ae2b-19b559064edb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:57:13.813170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.813144    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89e0f65a-5731-4b10-ae2b-19b559064edb-kube-api-access-7qwbg" pod "89e0f65a-5731-4b10-ae2b-19b559064edb" (UID: "89e0f65a-5731-4b10-ae2b-19b559064edb"). InnerVolumeSpecName "kube-api-access-7qwbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:13.912395 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.912356    4925 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e0f65a-5731-4b10-ae2b-19b559064edb-tls-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:13.912395 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.912396    4925 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-dshm\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:13.912684 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.912412    4925 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-model-cache\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:13.912684 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.912427    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qwbg\" (UniqueName: \"kubernetes.io/projected/89e0f65a-5731-4b10-ae2b-19b559064edb-kube-api-access-7qwbg\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:13.912684 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.912440    4925 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-home\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:13.912684 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:13.912476    4925 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89e0f65a-5731-4b10-ae2b-19b559064edb-tmp-dir\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:14.336171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.335964    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825" podUID="425dbdc1-8cf7-40b5-b019-c245612eed41" containerName="payload-pre-processing" containerID="cri-o://bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f" gracePeriod=25
Oct 30 18:57:14.345488 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.345295    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg" podUID="09d5a17a-9dbb-4f29-b3d7-3a807e013a9d" containerName="payload-processing" containerID="cri-o://d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c" gracePeriod=25
Oct 30 18:57:14.360958 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.358754    4925 generic.go:356] "Generic (PLEG): container finished" podID="89e0f65a-5731-4b10-ae2b-19b559064edb" containerID="24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831" exitCode=0
Oct 30 18:57:14.360958 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.358976    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp" event={"ID":"89e0f65a-5731-4b10-ae2b-19b559064edb","Type":"ContainerDied","Data":"24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831"}
Oct 30 18:57:14.360958 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.359011    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp" event={"ID":"89e0f65a-5731-4b10-ae2b-19b559064edb","Type":"ContainerDied","Data":"5c3ac632d44d168ff08210689dccbfaa52692e9d20625fe7bb0f27ec9a09b02e"}
Oct 30 18:57:14.360958 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.359032    4925 scope.go:122] "RemoveContainer" containerID="24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831"
Oct 30 18:57:14.360958 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.359271    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"
Oct 30 18:57:14.490950 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.490923    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"]
Oct 30 18:57:14.496766 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.496669    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-da58b769/e2e-lifecycle-model-da58b769-kserve-57bd7d5dc7-wcfqp"]
Oct 30 18:57:14.499539 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.499273    4925 scope.go:122] "RemoveContainer" containerID="24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831"
Oct 30 18:57:14.499840 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:57:14.499746    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831\": container with ID starting with 24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831 not found: ID does not exist" containerID="24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831"
Oct 30 18:57:14.499840 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.499781    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831"} err="failed to get container status \"24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831\": rpc error: code = NotFound desc = could not find container \"24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831\": container with ID starting with 24ebd1f259170dff208619b50d25cc123617ab63981af6da6624bb03549f6831 not found: ID does not exist"
Oct 30 18:57:14.761144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.749878    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:14.822892 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.822666    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"
Oct 30 18:57:14.826521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.826428    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"
Oct 30 18:57:14.969914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.969886    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/425dbdc1-8cf7-40b5-b019-c245612eed41-kube-api-access-rlfhb\" (UniqueName: \"kubernetes.io/projected/425dbdc1-8cf7-40b5-b019-c245612eed41-kube-api-access-rlfhb\") pod \"425dbdc1-8cf7-40b5-b019-c245612eed41\" (UID: \"425dbdc1-8cf7-40b5-b019-c245612eed41\") "
Oct 30 18:57:14.970280 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.970258    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/425dbdc1-8cf7-40b5-b019-c245612eed41-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/425dbdc1-8cf7-40b5-b019-c245612eed41-plugins-config-volume\") pod \"425dbdc1-8cf7-40b5-b019-c245612eed41\" (UID: \"425dbdc1-8cf7-40b5-b019-c245612eed41\") "
Oct 30 18:57:14.970412 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.970366    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-kube-api-access-ncdjb\" (UniqueName: \"kubernetes.io/projected/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-kube-api-access-ncdjb\") pod \"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d\" (UID: \"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d\") "
Oct 30 18:57:14.970474 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.970438    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-plugins-config-volume\") pod \"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d\" (UID: \"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d\") "
Oct 30 18:57:14.970949 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.970802    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-plugins-config-volume" pod "09d5a17a-9dbb-4f29-b3d7-3a807e013a9d" (UID: "09d5a17a-9dbb-4f29-b3d7-3a807e013a9d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:14.971075 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.970990    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/425dbdc1-8cf7-40b5-b019-c245612eed41-plugins-config-volume" pod "425dbdc1-8cf7-40b5-b019-c245612eed41" (UID: "425dbdc1-8cf7-40b5-b019-c245612eed41"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:14.971379 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.971353    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/425dbdc1-8cf7-40b5-b019-c245612eed41-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:14.971452 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.971386    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:14.972642 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.972609    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/425dbdc1-8cf7-40b5-b019-c245612eed41-kube-api-access-rlfhb" pod "425dbdc1-8cf7-40b5-b019-c245612eed41" (UID: "425dbdc1-8cf7-40b5-b019-c245612eed41"). InnerVolumeSpecName "kube-api-access-rlfhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:14.973471 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:14.973444    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-kube-api-access-ncdjb" pod "09d5a17a-9dbb-4f29-b3d7-3a807e013a9d" (UID: "09d5a17a-9dbb-4f29-b3d7-3a807e013a9d"). InnerVolumeSpecName "kube-api-access-ncdjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:15.073263 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.073181    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlfhb\" (UniqueName: \"kubernetes.io/projected/425dbdc1-8cf7-40b5-b019-c245612eed41-kube-api-access-rlfhb\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:15.073263 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.073211    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncdjb\" (UniqueName: \"kubernetes.io/projected/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d-kube-api-access-ncdjb\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:15.415609 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.415476    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:15.438209 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.435127    4925 generic.go:356] "Generic (PLEG): container finished" podID="425dbdc1-8cf7-40b5-b019-c245612eed41" containerID="bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f" exitCode=0
Oct 30 18:57:15.438209 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.435363    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825" event={"ID":"425dbdc1-8cf7-40b5-b019-c245612eed41","Type":"ContainerDied","Data":"bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f"}
Oct 30 18:57:15.438209 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.435417    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825" event={"ID":"425dbdc1-8cf7-40b5-b019-c245612eed41","Type":"ContainerDied","Data":"587bf527fc67e5ff1f952097f040c0b57792e6996e8ae302eebe6431eace4d64"}
Oct 30 18:57:15.438209 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.435448    4925 scope.go:122] "RemoveContainer" containerID="bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f"
Oct 30 18:57:15.438209 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.435551    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"
Oct 30 18:57:15.447946 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.447913    4925 generic.go:356] "Generic (PLEG): container finished" podID="09d5a17a-9dbb-4f29-b3d7-3a807e013a9d" containerID="d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c" exitCode=0
Oct 30 18:57:15.448264 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.448242    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg" event={"ID":"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d","Type":"ContainerDied","Data":"d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c"}
Oct 30 18:57:15.448428 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.448410    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg" event={"ID":"09d5a17a-9dbb-4f29-b3d7-3a807e013a9d","Type":"ContainerDied","Data":"7717ed193095d930cc78a3b8ac2ddbff1262a718fb0581fc8c776da73592b985"}
Oct 30 18:57:15.448605 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.448588    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"
Oct 30 18:57:15.492194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.492167    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"]
Oct 30 18:57:15.492952 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.492930    4925 scope.go:122] "RemoveContainer" containerID="bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f"
Oct 30 18:57:15.493558 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:57:15.493531    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f\": container with ID starting with bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f not found: ID does not exist" containerID="bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f"
Oct 30 18:57:15.493649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.493568    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f"} err="failed to get container status \"bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f\": rpc error: code = NotFound desc = could not find container \"bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f\": container with ID starting with bbca3781fdd0feb4d32ab9b22cd6d25fff15d8bff02f101f3c44bda38534219f not found: ID does not exist"
Oct 30 18:57:15.493649 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.493592    4925 scope.go:122] "RemoveContainer" containerID="d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c"
Oct 30 18:57:15.495301 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.495276    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-8f1c37-6fff6dcbb7-vz825"]
Oct 30 18:57:15.502653 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.502632    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"]
Oct 30 18:57:15.506834 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.506814    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-8f1c37-677b95b7d-6q6qg"]
Oct 30 18:57:15.511751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.511731    4925 scope.go:122] "RemoveContainer" containerID="d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c"
Oct 30 18:57:15.512143 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:57:15.512074    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c\": container with ID starting with d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c not found: ID does not exist" containerID="d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c"
Oct 30 18:57:15.512220 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:15.512155    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c"} err="failed to get container status \"d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c\": rpc error: code = NotFound desc = could not find container \"d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c\": container with ID starting with d7019b0ddd3566c2ada48871ae8ca28aa00481c73c5ebbf425e3759abfabc53c not found: ID does not exist"
Oct 30 18:57:16.037692 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:16.037652    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:16.236401 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:16.236362    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:16.425011 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:16.424978    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="09d5a17a-9dbb-4f29-b3d7-3a807e013a9d" path="/var/lib/kubelet/pods/09d5a17a-9dbb-4f29-b3d7-3a807e013a9d/volumes"
Oct 30 18:57:16.429014 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:16.425478    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="425dbdc1-8cf7-40b5-b019-c245612eed41" path="/var/lib/kubelet/pods/425dbdc1-8cf7-40b5-b019-c245612eed41/volumes"
Oct 30 18:57:16.429014 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:16.426004    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89e0f65a-5731-4b10-ae2b-19b559064edb" path="/var/lib/kubelet/pods/89e0f65a-5731-4b10-ae2b-19b559064edb/volumes"
Oct 30 18:57:16.583503 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:16.561719    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:16.842050 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:16.841956    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:17.162833 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:17.162298    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"
Oct 30 18:57:17.178407 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:17.178376    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"
Oct 30 18:57:17.182538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:17.182509    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:17.688627 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:17.688580    4925 generic.go:356] "Generic (PLEG): container finished" podID="17503242-eada-4ef5-b87a-8c766b072494" containerID="da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192" exitCode=0
Oct 30 18:57:17.689030 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:17.688825    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" event={"ID":"17503242-eada-4ef5-b87a-8c766b072494","Type":"ContainerDied","Data":"da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192"}
Oct 30 18:57:17.689277 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:17.689257    4925 scope.go:122] "RemoveContainer" containerID="da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192"
Oct 30 18:57:17.690873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:17.690835    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:57:17.690988 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:17.690935    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:57:17.691049 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:17.690990    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:57:18.078812 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:18.078715    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:18.424333 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:18.424298    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:57:18.701029 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:18.700924    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" event={"ID":"17503242-eada-4ef5-b87a-8c766b072494","Type":"ContainerStarted","Data":"354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300"}
Oct 30 18:57:18.744561 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:18.744530    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:57:18.745578 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:18.745542    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 18:57:18.745709 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:18.745594    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 18:57:19.175320 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:19.169876    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:19.745524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:19.745487    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 18:57:19.745910 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:19.745548    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 18:57:19.751011 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:19.750983    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:19.935384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:19.935350    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:20.454905 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.454728    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:20.745345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.745246    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 18:57:20.745345 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.745305    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 18:57:20.928062 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.928014    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"]
Oct 30 18:57:20.929415 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.929395    4925 cpu_manager.go:414] "containerMap: removing container" podUID="09d5a17a-9dbb-4f29-b3d7-3a807e013a9d" containerName="payload-processing"
Oct 30 18:57:20.929415 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.929416    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="09d5a17a-9dbb-4f29-b3d7-3a807e013a9d" containerName="payload-processing"
Oct 30 18:57:20.929538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.929466    4925 cpu_manager.go:414] "containerMap: removing container" podUID="89e0f65a-5731-4b10-ae2b-19b559064edb" containerName="main"
Oct 30 18:57:20.929538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.929471    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89e0f65a-5731-4b10-ae2b-19b559064edb" containerName="main"
Oct 30 18:57:20.929538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.929493    4925 cpu_manager.go:414] "containerMap: removing container" podUID="425dbdc1-8cf7-40b5-b019-c245612eed41" containerName="payload-pre-processing"
Oct 30 18:57:20.929538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.929499    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="425dbdc1-8cf7-40b5-b019-c245612eed41" containerName="payload-pre-processing"
Oct 30 18:57:20.929681 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.929630    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="09d5a17a-9dbb-4f29-b3d7-3a807e013a9d" containerName="payload-processing"
Oct 30 18:57:20.929681 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.929646    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="89e0f65a-5731-4b10-ae2b-19b559064edb" containerName="main"
Oct 30 18:57:20.929681 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.929655    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="425dbdc1-8cf7-40b5-b019-c245612eed41" containerName="payload-pre-processing"
Oct 30 18:57:20.935503 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.935131    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"
Oct 30 18:57:20.941729 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.940341    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"]
Oct 30 18:57:20.944015 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.943993    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-54bc2b\""
Oct 30 18:57:20.994206 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.994172    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:20.999174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:20.997872    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"]
Oct 30 18:57:21.006161 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.005392    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"
Oct 30 18:57:21.009716 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.009692    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-54bc2b-dockercfg-n5gfv\""
Oct 30 18:57:21.009716 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.009694    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"]
Oct 30 18:57:21.156122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.152164    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc97dc0-498f-419e-bfd1-072b72fb8658-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd\" (UID: \"bdc97dc0-498f-419e-bfd1-072b72fb8658\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"
Oct 30 18:57:21.156122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.152250    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n\" (UID: \"01d7f5f2-df5b-44c0-a81b-3fe176212a1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"
Oct 30 18:57:21.156122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.152371    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gnns\" (UniqueName: \"kubernetes.io/projected/bdc97dc0-498f-419e-bfd1-072b72fb8658-kube-api-access-7gnns\") pod \"payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd\" (UID: \"bdc97dc0-498f-419e-bfd1-072b72fb8658\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"
Oct 30 18:57:21.156122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.152509    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqrbd\" (UniqueName: \"kubernetes.io/projected/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-kube-api-access-hqrbd\") pod \"payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n\" (UID: \"01d7f5f2-df5b-44c0-a81b-3fe176212a1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"
Oct 30 18:57:21.236117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.229864    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:21.257167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.254014    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc97dc0-498f-419e-bfd1-072b72fb8658-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd\" (UID: \"bdc97dc0-498f-419e-bfd1-072b72fb8658\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"
Oct 30 18:57:21.257167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.254111    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n\" (UID: \"01d7f5f2-df5b-44c0-a81b-3fe176212a1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"
Oct 30 18:57:21.257167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.254200    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gnns\" (UniqueName: \"kubernetes.io/projected/bdc97dc0-498f-419e-bfd1-072b72fb8658-kube-api-access-7gnns\") pod \"payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd\" (UID: \"bdc97dc0-498f-419e-bfd1-072b72fb8658\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"
Oct 30 18:57:21.257167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.254317    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqrbd\" (UniqueName: \"kubernetes.io/projected/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-kube-api-access-hqrbd\") pod \"payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n\" (UID: \"01d7f5f2-df5b-44c0-a81b-3fe176212a1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"
Oct 30 18:57:21.257167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.255530    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc97dc0-498f-419e-bfd1-072b72fb8658-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd\" (UID: \"bdc97dc0-498f-419e-bfd1-072b72fb8658\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"
Oct 30 18:57:21.257167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.256281    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n\" (UID: \"01d7f5f2-df5b-44c0-a81b-3fe176212a1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"
Oct 30 18:57:21.324330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.324291    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gnns\" (UniqueName: \"kubernetes.io/projected/bdc97dc0-498f-419e-bfd1-072b72fb8658-kube-api-access-7gnns\") pod \"payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd\" (UID: \"bdc97dc0-498f-419e-bfd1-072b72fb8658\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"
Oct 30 18:57:21.339728 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.339664    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqrbd\" (UniqueName: \"kubernetes.io/projected/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-kube-api-access-hqrbd\") pod \"payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n\" (UID: \"01d7f5f2-df5b-44c0-a81b-3fe176212a1f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"
Oct 30 18:57:21.604547 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.604467    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"
Oct 30 18:57:21.625476 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.625446    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"
Oct 30 18:57:21.745473 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.745437    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 18:57:21.745644 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.745502    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 18:57:21.944123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:21.936631    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:22.024478 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.021354    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"]
Oct 30 18:57:22.028435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.026713    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"]
Oct 30 18:57:22.028435 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:57:22.027914    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d7f5f2_df5b_44c0_a81b_3fe176212a1f.slice/crio-9f9dc91c9b6d03c9421b79ec5e3fd6de4a94791d40f5a8f02ea5f7070a4d2234 WatchSource:0}: Error finding container 9f9dc91c9b6d03c9421b79ec5e3fd6de4a94791d40f5a8f02ea5f7070a4d2234: Status 404 returned error can't find the container with id 9f9dc91c9b6d03c9421b79ec5e3fd6de4a94791d40f5a8f02ea5f7070a4d2234
Oct 30 18:57:22.038381 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:57:22.038357    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdc97dc0_498f_419e_bfd1_072b72fb8658.slice/crio-74fd1cefbd15c6183ba83cc70f8b8d3fd17cd126b91a0e243fd48ea93eb7f24d WatchSource:0}: Error finding container 74fd1cefbd15c6183ba83cc70f8b8d3fd17cd126b91a0e243fd48ea93eb7f24d: Status 404 returned error can't find the container with id 74fd1cefbd15c6183ba83cc70f8b8d3fd17cd126b91a0e243fd48ea93eb7f24d
Oct 30 18:57:22.292248 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.292027    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"
Oct 30 18:57:22.292411 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.292363    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"
Oct 30 18:57:22.751239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.751204    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 18:57:22.781619 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.781580    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 18:57:22.799160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.778143    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n" event={"ID":"01d7f5f2-df5b-44c0-a81b-3fe176212a1f","Type":"ContainerStarted","Data":"9eb44fbbf2bfc5bd7d5471e7f040357d0c6581b786dd11666ce9dd9e14cd1109"}
Oct 30 18:57:22.799160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.799018    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n" event={"ID":"01d7f5f2-df5b-44c0-a81b-3fe176212a1f","Type":"ContainerStarted","Data":"9f9dc91c9b6d03c9421b79ec5e3fd6de4a94791d40f5a8f02ea5f7070a4d2234"}
Oct 30 18:57:22.799160 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.799123    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"
Oct 30 18:57:22.799488 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.799377    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:22.822207 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.822173    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd" event={"ID":"bdc97dc0-498f-419e-bfd1-072b72fb8658","Type":"ContainerStarted","Data":"a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9"}
Oct 30 18:57:22.822478 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.822456    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd" event={"ID":"bdc97dc0-498f-419e-bfd1-072b72fb8658","Type":"ContainerStarted","Data":"74fd1cefbd15c6183ba83cc70f8b8d3fd17cd126b91a0e243fd48ea93eb7f24d"}
Oct 30 18:57:22.822887 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.822866    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"
Oct 30 18:57:22.887951 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.887869    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n" podStartSLOduration=2.887845572 podStartE2EDuration="2.887845572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:22.852581542 +0000 UTC m=+1264.568926619" watchObservedRunningTime="2026-10-30 18:57:22.887845572 +0000 UTC m=+1264.604190651"
Oct 30 18:57:22.925474 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:22.925417    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd" podStartSLOduration=2.925402439 podStartE2EDuration="2.925402439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:22.87439992 +0000 UTC m=+1264.590744997" watchObservedRunningTime="2026-10-30 18:57:22.925402439 +0000 UTC m=+1264.641747514"
Oct 30 18:57:23.593121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:23.586075    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:23.745050 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:23.745011    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 18:57:23.745273 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:23.745085    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 18:57:24.264414 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:24.264379    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-vfsrt_77704542-8e8e-4d3e-a4bd-e07d4f0e79a0/payload-processing/0.log"
Oct 30 18:57:24.372055 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:24.369122    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:24.487559 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:24.487531    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-vfsrt_77704542-8e8e-4d3e-a4bd-e07d4f0e79a0/payload-processing/0.log"
Oct 30 18:57:24.661933 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:24.661905    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6_c7eea034-caf9-4b41-8cbb-77715993d9d1/payload-processing/0.log"
Oct 30 18:57:24.749654 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:24.749624    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:57:24.750086 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:24.750071    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:57:24.750815 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:24.750794    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:57:25.148074 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:25.147994    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:25.722616 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:25.722578    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:25.917130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:25.916602    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:26.308830 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:26.308798    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:26.495120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:26.495072    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:27.116990 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.116956    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:27.120169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.120139    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"]
Oct 30 18:57:27.158676 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.158651    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.168117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.163419    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-93574a\"/\"default-dockercfg-89qdp\""
Oct 30 18:57:27.172971 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.172590    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-93574a\"/\"openshift-service-ca.crt\""
Oct 30 18:57:27.174138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.173810    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-93574a\"/\"ipp-route-93574a-kserve-self-signed-certs\""
Oct 30 18:57:27.174138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.173993    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-93574a\"/\"kube-root-ca.crt\""
Oct 30 18:57:27.185264 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.179142    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"]
Oct 30 18:57:27.264136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.259979    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-model-cache\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.264136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.260080    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-dshm\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.264136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.260166    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tls-certs\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.264136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.260295    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-home\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.264136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.260402    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mzhn\" (UniqueName: \"kubernetes.io/projected/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-kube-api-access-8mzhn\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.264136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.260431    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tmp-dir\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.327374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.327058    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:27.362558 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.362525    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-home\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.362845 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.362086    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-home\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.367271 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.363048    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mzhn\" (UniqueName: \"kubernetes.io/projected/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-kube-api-access-8mzhn\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.392216 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.392189    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tmp-dir\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.392868 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.392758    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tmp-dir\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.394025 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.393200    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-model-cache\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.394317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.394271    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-dshm\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.398624 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.398602    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tls-certs\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.403334 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.402963    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mzhn\" (UniqueName: \"kubernetes.io/projected/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-kube-api-access-8mzhn\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.403499 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.393476    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-model-cache\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.427125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.426466    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-dshm\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.445127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.441719    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tls-certs\") pod \"ipp-route-93574a-kserve-8474dbd557-xvhmm\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") " pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.503510 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.503473    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:27.806691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.806659    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"]
Oct 30 18:57:27.809662 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:57:27.809632    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47fba953_aa96_4b41_9cc5_f00d6c6b2ffb.slice/crio-309ec77ea33b304db9e0f5af9ebb42bc4dbe459dbb9e2a3517638a4ca0978221 WatchSource:0}: Error finding container 309ec77ea33b304db9e0f5af9ebb42bc4dbe459dbb9e2a3517638a4ca0978221: Status 404 returned error can't find the container with id 309ec77ea33b304db9e0f5af9ebb42bc4dbe459dbb9e2a3517638a4ca0978221
Oct 30 18:57:27.947198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.947153    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm" event={"ID":"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb","Type":"ContainerStarted","Data":"8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6"}
Oct 30 18:57:27.947198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.947196    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm" event={"ID":"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb","Type":"ContainerStarted","Data":"309ec77ea33b304db9e0f5af9ebb42bc4dbe459dbb9e2a3517638a4ca0978221"}
Oct 30 18:57:27.970923 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:27.970869    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm" podStartSLOduration=0.970851744 podStartE2EDuration="970.851744ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:27.968696607 +0000 UTC m=+1269.685041684" watchObservedRunningTime="2026-10-30 18:57:27.970851744 +0000 UTC m=+1269.687196820"
Oct 30 18:57:28.210934 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:28.210895    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:28.503923 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:28.503840    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:28.874247 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:28.865946    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:29.707319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:29.706745    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:29.889468 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:29.889379    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:30.064184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:30.059944    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:30.591124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:30.589649    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:31.446171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:31.443535    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:32.604122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:32.603754    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:33.110123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:33.104700    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:33.411976 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:33.411943    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:33.638126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:33.636704    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:33.837510 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:33.837419    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"
Oct 30 18:57:33.837668 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:33.837558    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"
Oct 30 18:57:34.494120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:34.487302    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:34.764405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:34.764289    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:35.275121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:35.273619    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:36.017121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:36.015605    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:36.189832 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:36.189779    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:36.666555 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:36.658711    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:37.418438 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:37.414690    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:37.504215 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:37.504184    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:37.517458 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:37.517432    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:38.097603 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:38.097565    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:38.154220 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:38.153981    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:57:38.594893 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:38.594852    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"]
Oct 30 18:57:38.595413 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:38.595130    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5" podUID="69c1f490-4b34-47d5-bae8-b7e6c3bb67ae" containerName="maas-api" containerID="cri-o://b662941d8439580fb1fdedca49923668374c2c30c67463480c10aebd17ebcb67" gracePeriod=30
Oct 30 18:57:38.655883 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:38.655850    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"]
Oct 30 18:57:38.656324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:38.656078    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2" podUID="bda1aecc-c6a3-4d7d-ad62-f78fb46264ba" containerName="payload-processing" containerID="cri-o://f6f4b8d70c614840380e2e6fa361d461bde0e51c30be60b6c7a4a66c795cbad4" gracePeriod=30
Oct 30 18:57:38.680245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:38.680217    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"]
Oct 30 18:57:38.680637 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:38.680603    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v" podUID="f29a9a95-cd18-4a60-a38b-c83b77cfadf2" containerName="payload-pre-processing" containerID="cri-o://935edee7c27ce5b6879b9654bd194103f6568a7ad497e104d9e66ff33fc8247e" gracePeriod=30
Oct 30 18:57:38.731060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:38.730922    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:38.935121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:38.929606    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:39.109878 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.090699    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5" podUID="69c1f490-4b34-47d5-bae8-b7e6c3bb67ae" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.107:8443/health\": dial tcp 10.132.0.107:8443: connect: connection refused"
Oct 30 18:57:39.127086 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.126609    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:39.154045 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.142318    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v" podUID="f29a9a95-cd18-4a60-a38b-c83b77cfadf2" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.112:9004: connect: connection refused"
Oct 30 18:57:39.199172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.190398    4925 generic.go:356] "Generic (PLEG): container finished" podID="69c1f490-4b34-47d5-bae8-b7e6c3bb67ae" containerID="b662941d8439580fb1fdedca49923668374c2c30c67463480c10aebd17ebcb67" exitCode=0
Oct 30 18:57:39.199172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.190544    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5" event={"ID":"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae","Type":"ContainerDied","Data":"b662941d8439580fb1fdedca49923668374c2c30c67463480c10aebd17ebcb67"}
Oct 30 18:57:39.199172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.192599    4925 generic.go:356] "Generic (PLEG): container finished" podID="f29a9a95-cd18-4a60-a38b-c83b77cfadf2" containerID="935edee7c27ce5b6879b9654bd194103f6568a7ad497e104d9e66ff33fc8247e" exitCode=0
Oct 30 18:57:39.199172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.192738    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v" event={"ID":"f29a9a95-cd18-4a60-a38b-c83b77cfadf2","Type":"ContainerDied","Data":"935edee7c27ce5b6879b9654bd194103f6568a7ad497e104d9e66ff33fc8247e"}
Oct 30 18:57:39.199172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.195171    4925 generic.go:356] "Generic (PLEG): container finished" podID="bda1aecc-c6a3-4d7d-ad62-f78fb46264ba" containerID="f6f4b8d70c614840380e2e6fa361d461bde0e51c30be60b6c7a4a66c795cbad4" exitCode=0
Oct 30 18:57:39.199172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.196328    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2" event={"ID":"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba","Type":"ContainerDied","Data":"f6f4b8d70c614840380e2e6fa361d461bde0e51c30be60b6c7a4a66c795cbad4"}
Oct 30 18:57:39.418854 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.418814    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:39.502274 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.500935    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"
Oct 30 18:57:39.521873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.521555    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:57:39.533150 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.533119    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"
Oct 30 18:57:39.593864 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.593827    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:39.603968 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.603942    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-kube-api-access-wjbsp\" (UniqueName: \"kubernetes.io/projected/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-kube-api-access-wjbsp\") pod \"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba\" (UID: \"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba\") "
Oct 30 18:57:39.604455 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.604237    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-plugins-config-volume\") pod \"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba\" (UID: \"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba\") "
Oct 30 18:57:39.604617 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.604590    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-plugins-config-volume" pod "bda1aecc-c6a3-4d7d-ad62-f78fb46264ba" (UID: "bda1aecc-c6a3-4d7d-ad62-f78fb46264ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:39.605036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.605013    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:39.606456 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.606433    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-kube-api-access-wjbsp" pod "bda1aecc-c6a3-4d7d-ad62-f78fb46264ba" (UID: "bda1aecc-c6a3-4d7d-ad62-f78fb46264ba"). InnerVolumeSpecName "kube-api-access-wjbsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:39.705624 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.705585    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-maas-api-tls\") pod \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") "
Oct 30 18:57:39.705816 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.705656    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-plugins-config-volume\") pod \"f29a9a95-cd18-4a60-a38b-c83b77cfadf2\" (UID: \"f29a9a95-cd18-4a60-a38b-c83b77cfadf2\") "
Oct 30 18:57:39.705816 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.705731    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-openshift-service-ca\") pod \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") "
Oct 30 18:57:39.705942 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.705862    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-metrics-certs\") pod \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") "
Oct 30 18:57:39.705942 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.705945    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-kube-api-access-bx7tg\" (UniqueName: \"kubernetes.io/projected/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-kube-api-access-bx7tg\") pod \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\" (UID: \"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae\") "
Oct 30 18:57:39.706138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.706043    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-kube-api-access-hg29z\" (UniqueName: \"kubernetes.io/projected/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-kube-api-access-hg29z\") pod \"f29a9a95-cd18-4a60-a38b-c83b77cfadf2\" (UID: \"f29a9a95-cd18-4a60-a38b-c83b77cfadf2\") "
Oct 30 18:57:39.706138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.706114    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-openshift-service-ca" pod "69c1f490-4b34-47d5-bae8-b7e6c3bb67ae" (UID: "69c1f490-4b34-47d5-bae8-b7e6c3bb67ae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:39.706256 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.706132    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-plugins-config-volume" pod "f29a9a95-cd18-4a60-a38b-c83b77cfadf2" (UID: "f29a9a95-cd18-4a60-a38b-c83b77cfadf2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:39.707171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.707146    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:39.707271 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.707176    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:39.707271 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.707193    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjbsp\" (UniqueName: \"kubernetes.io/projected/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba-kube-api-access-wjbsp\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:39.708363 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.708339    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-metrics-certs" pod "69c1f490-4b34-47d5-bae8-b7e6c3bb67ae" (UID: "69c1f490-4b34-47d5-bae8-b7e6c3bb67ae"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:57:39.708768 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.708745    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-maas-api-tls" pod "69c1f490-4b34-47d5-bae8-b7e6c3bb67ae" (UID: "69c1f490-4b34-47d5-bae8-b7e6c3bb67ae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:57:39.708926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.708846    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-kube-api-access-bx7tg" pod "69c1f490-4b34-47d5-bae8-b7e6c3bb67ae" (UID: "69c1f490-4b34-47d5-bae8-b7e6c3bb67ae"). InnerVolumeSpecName "kube-api-access-bx7tg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:39.709510 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.709483    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-kube-api-access-hg29z" pod "f29a9a95-cd18-4a60-a38b-c83b77cfadf2" (UID: "f29a9a95-cd18-4a60-a38b-c83b77cfadf2"). InnerVolumeSpecName "kube-api-access-hg29z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:39.768440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.768414    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:39.808673 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.808638    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:39.808805 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.808675    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:39.808805 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.808693    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx7tg\" (UniqueName: \"kubernetes.io/projected/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae-kube-api-access-bx7tg\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:39.808805 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:39.808702    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg29z\" (UniqueName: \"kubernetes.io/projected/f29a9a95-cd18-4a60-a38b-c83b77cfadf2-kube-api-access-hg29z\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:40.159177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.154281    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2" podUID="bda1aecc-c6a3-4d7d-ad62-f78fb46264ba" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.113:9004: i/o timeout"
Oct 30 18:57:40.237121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.236989    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2" event={"ID":"bda1aecc-c6a3-4d7d-ad62-f78fb46264ba","Type":"ContainerDied","Data":"e3033f3916b66c2760b052609f808d68e570e1559a0c802ec1c71134ff885988"}
Oct 30 18:57:40.237324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.237205    4925 scope.go:122] "RemoveContainer" containerID="f6f4b8d70c614840380e2e6fa361d461bde0e51c30be60b6c7a4a66c795cbad4"
Oct 30 18:57:40.237543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.237398    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"
Oct 30 18:57:40.251020 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.250985    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5" event={"ID":"69c1f490-4b34-47d5-bae8-b7e6c3bb67ae","Type":"ContainerDied","Data":"1fe137af585cf153fb13cf6dbbe873e020ce13288d3c5dd0ef13a0553bea9c71"}
Oct 30 18:57:40.251341 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.251306    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"
Oct 30 18:57:40.257704 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.257675    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v" event={"ID":"f29a9a95-cd18-4a60-a38b-c83b77cfadf2","Type":"ContainerDied","Data":"5fc42657e921edb532d119aec7be84222817e70c8f0fdfac8b4bb3788419265e"}
Oct 30 18:57:40.257942 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.257909    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"
Oct 30 18:57:40.324518 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.324492    4925 scope.go:122] "RemoveContainer" containerID="b662941d8439580fb1fdedca49923668374c2c30c67463480c10aebd17ebcb67"
Oct 30 18:57:40.349784 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.349755    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"]
Oct 30 18:57:40.365271 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.365239    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-040d9c-b97b64597-dmtm5"]
Oct 30 18:57:40.377732 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.377201    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"]
Oct 30 18:57:40.393200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.392958    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-040d9c-598f4c788b-r7xx2"]
Oct 30 18:57:40.402352 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.402334    4925 scope.go:122] "RemoveContainer" containerID="935edee7c27ce5b6879b9654bd194103f6568a7ad497e104d9e66ff33fc8247e"
Oct 30 18:57:40.413315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.413254    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"]
Oct 30 18:57:40.425388 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.425362    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69c1f490-4b34-47d5-bae8-b7e6c3bb67ae" path="/var/lib/kubelet/pods/69c1f490-4b34-47d5-bae8-b7e6c3bb67ae/volumes"
Oct 30 18:57:40.425857 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.425841    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bda1aecc-c6a3-4d7d-ad62-f78fb46264ba" path="/var/lib/kubelet/pods/bda1aecc-c6a3-4d7d-ad62-f78fb46264ba/volumes"
Oct 30 18:57:40.426172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.426157    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-040d9c-9848c8999-gk44v"]
Oct 30 18:57:40.617123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:40.614938    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:41.549154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:41.548591    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:42.433120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:42.432592    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:42.437118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:42.434314    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f29a9a95-cd18-4a60-a38b-c83b77cfadf2" path="/var/lib/kubelet/pods/f29a9a95-cd18-4a60-a38b-c83b77cfadf2/volumes"
Oct 30 18:57:42.620150 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:42.620108    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:42.813815 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:42.813719    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:42.890194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:42.881546    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"]
Oct 30 18:57:42.890194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:42.881838    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x" podUID="1aa78826-8a83-4c64-b87e-642678bf9774" containerName="istio-proxy" containerID="cri-o://f6fdc76ecc10a2b164858fd8a6e0509acff9b518212ab8c4a3671a5a287f3f7e" gracePeriod=30
Oct 30 18:57:42.920152 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:42.912257    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"]
Oct 30 18:57:42.920152 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:42.912518    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2" podUID="32f50b9e-f1e8-47e9-8e50-e3ba166af2de" containerName="istio-proxy" containerID="cri-o://bf4ec6f99c51191b869e5fa80d3d6a2529efcea12f0da827ba3cf0082c2e8c3f" gracePeriod=30
Oct 30 18:57:42.922504 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:42.920745    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"]
Oct 30 18:57:42.922504 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:42.922185    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy" containerID="cri-o://0ca7a4a6c2d2208f156e85652e4248cb2749b8a70f781d2fd3727804df5e3cce" gracePeriod=30
Oct 30 18:57:43.318797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:43.318765    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:43.341698 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:57:43.341665    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:57:44.144122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.134601    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:44.183187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.180819    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"]
Oct 30 18:57:44.318963 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.318917    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:44.589163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.588700    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:44.727124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.718117    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"]
Oct 30 18:57:44.742163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.740348    4925 cpu_manager.go:414] "containerMap: removing container" podUID="bda1aecc-c6a3-4d7d-ad62-f78fb46264ba" containerName="payload-processing"
Oct 30 18:57:44.742163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.740374    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bda1aecc-c6a3-4d7d-ad62-f78fb46264ba" containerName="payload-processing"
Oct 30 18:57:44.742163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.740408    4925 cpu_manager.go:414] "containerMap: removing container" podUID="f29a9a95-cd18-4a60-a38b-c83b77cfadf2" containerName="payload-pre-processing"
Oct 30 18:57:44.742163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.740416    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f29a9a95-cd18-4a60-a38b-c83b77cfadf2" containerName="payload-pre-processing"
Oct 30 18:57:44.742163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.740502    4925 cpu_manager.go:414] "containerMap: removing container" podUID="69c1f490-4b34-47d5-bae8-b7e6c3bb67ae" containerName="maas-api"
Oct 30 18:57:44.742163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.740511    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69c1f490-4b34-47d5-bae8-b7e6c3bb67ae" containerName="maas-api"
Oct 30 18:57:44.742163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.740807    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="bda1aecc-c6a3-4d7d-ad62-f78fb46264ba" containerName="payload-processing"
Oct 30 18:57:44.742163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.740823    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="f29a9a95-cd18-4a60-a38b-c83b77cfadf2" containerName="payload-pre-processing"
Oct 30 18:57:44.742163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.740844    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="69c1f490-4b34-47d5-bae8-b7e6c3bb67ae" containerName="maas-api"
Oct 30 18:57:44.750797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.749478    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"]
Oct 30 18:57:44.750797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.749595    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:44.773924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.773360    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-6a327a-openshift-default-dockercfg-rft58\""
Oct 30 18:57:44.886869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.886780    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-credential-socket\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:44.913054 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.913013    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/824748a3-c26d-4699-a479-9a5be67f3c0d-istiod-ca-cert\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:44.913508 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.913481    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqm7q\" (UniqueName: \"kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-kube-api-access-lqm7q\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:44.913729 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.913713    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-certs\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:44.913892 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.913877    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-envoy\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:44.914147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.914128    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-data\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:44.914377 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.914362    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-token\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:44.914540 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.914525    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-socket\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:44.914716 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.914702    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-podinfo\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:44.914926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.895703    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:44.978291 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:44.965049    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"]
Oct 30 18:57:45.018252 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.018207    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/824748a3-c26d-4699-a479-9a5be67f3c0d-istiod-ca-cert\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.018580 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.018553    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqm7q\" (UniqueName: \"kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-kube-api-access-lqm7q\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.018767 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.018747    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-certs\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.018893 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.018877    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-envoy\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.019114 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.019078    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-data\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.019299 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.019277    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-token\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.019445 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.019429    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-socket\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.019613 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.019598    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-podinfo\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.019765 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.019750    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-credential-socket\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.020415 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.020393    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-credential-socket\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.029118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.021551    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-data\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.029828 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.029803    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-socket\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.105121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.092250    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/824748a3-c26d-4699-a479-9a5be67f3c0d-istiod-ca-cert\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.105121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.092567    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-certs\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.169624 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.169590    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqm7q\" (UniqueName: \"kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-kube-api-access-lqm7q\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.178350 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.178163    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"]
Oct 30 18:57:45.190060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.187170    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-podinfo\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.190060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.188220    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:45.193618 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.193595    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-envoy\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.194598 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.194575    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-token\") pod \"e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") " pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.387120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.378087    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:45.401528 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.394520    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:45.808243 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.807864    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:45.902798 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.902774    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"]
Oct 30 18:57:45.905365 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:57:45.905330    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod824748a3_c26d_4699_a479_9a5be67f3c0d.slice/crio-21cac42abce464833c9377b296cfea6fd0315cfa2d2f257f83809dd05dc7c29e WatchSource:0}: Error finding container 21cac42abce464833c9377b296cfea6fd0315cfa2d2f257f83809dd05dc7c29e: Status 404 returned error can't find the container with id 21cac42abce464833c9377b296cfea6fd0315cfa2d2f257f83809dd05dc7c29e
Oct 30 18:57:45.908110 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.908063    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:57:45.908204 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.908168    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:57:45.908243 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:45.908207    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:57:46.348379 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:46.348342    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" event={"ID":"824748a3-c26d-4699-a479-9a5be67f3c0d","Type":"ContainerStarted","Data":"fc13163f5124fd3b9c9b7b77d70ad5522ee0cc6640ee149c054b1529ec077c7c"}
Oct 30 18:57:46.348770 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:46.348402    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" event={"ID":"824748a3-c26d-4699-a479-9a5be67f3c0d","Type":"ContainerStarted","Data":"21cac42abce464833c9377b296cfea6fd0315cfa2d2f257f83809dd05dc7c29e"}
Oct 30 18:57:46.373931 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:46.373879    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" podStartSLOduration=2.373866309 podStartE2EDuration="2.373866309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:46.368129818 +0000 UTC m=+1288.084474899" watchObservedRunningTime="2026-10-30 18:57:46.373866309 +0000 UTC m=+1288.090211385"
Oct 30 18:57:46.397985 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:46.397953    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:46.404903 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:46.404882    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:46.575776 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:46.575740    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:46.765017 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:46.764983    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:47.033042 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:47.032956    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:47.357046 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:47.356961    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:47.359036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:47.359003    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:57:47.652079 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:47.651850    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:48.370237 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.370017    4925 generic.go:356] "Generic (PLEG): container finished" podID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerID="0ca7a4a6c2d2208f156e85652e4248cb2749b8a70f781d2fd3727804df5e3cce" exitCode=0
Oct 30 18:57:48.370607 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.370232    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" event={"ID":"49765651-6ef5-47c8-aef6-03b21952a2ad","Type":"ContainerDied","Data":"0ca7a4a6c2d2208f156e85652e4248cb2749b8a70f781d2fd3727804df5e3cce"}
Oct 30 18:57:48.370607 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.370311    4925 scope.go:122] "RemoveContainer" containerID="11512d0d37ba8bc25d3d68a508f18025278b9b8f2c345a3462ce9178403bb53d"
Oct 30 18:57:48.373982 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.373936    4925 generic.go:356] "Generic (PLEG): container finished" podID="32f50b9e-f1e8-47e9-8e50-e3ba166af2de" containerID="bf4ec6f99c51191b869e5fa80d3d6a2529efcea12f0da827ba3cf0082c2e8c3f" exitCode=0
Oct 30 18:57:48.374158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.374140    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2" event={"ID":"32f50b9e-f1e8-47e9-8e50-e3ba166af2de","Type":"ContainerDied","Data":"bf4ec6f99c51191b869e5fa80d3d6a2529efcea12f0da827ba3cf0082c2e8c3f"}
Oct 30 18:57:48.376947 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.376904    4925 generic.go:356] "Generic (PLEG): container finished" podID="1aa78826-8a83-4c64-b87e-642678bf9774" containerID="f6fdc76ecc10a2b164858fd8a6e0509acff9b518212ab8c4a3671a5a287f3f7e" exitCode=0
Oct 30 18:57:48.377589 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.377515    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x" event={"ID":"1aa78826-8a83-4c64-b87e-642678bf9774","Type":"ContainerDied","Data":"f6fdc76ecc10a2b164858fd8a6e0509acff9b518212ab8c4a3671a5a287f3f7e"}
Oct 30 18:57:48.476162 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.476135    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:57:48.493668 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.493636    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-credential-socket\") pod \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") "
Oct 30 18:57:48.493826 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.493697    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-token\" (UniqueName: \"kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-token\") pod \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") "
Oct 30 18:57:48.493826 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.493757    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istiod-ca-cert\") pod \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") "
Oct 30 18:57:48.493957 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.493903    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-socket\") pod \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") "
Oct 30 18:57:48.494043 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.493953    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-data\") pod \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") "
Oct 30 18:57:48.494043 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.494009    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-certs\") pod \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") "
Oct 30 18:57:48.494196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.494069    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-kube-api-access-t9tbk\" (UniqueName: \"kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-kube-api-access-t9tbk\") pod \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") "
Oct 30 18:57:48.494196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.494158    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-podinfo\") pod \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") "
Oct 30 18:57:48.494314 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.494246    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-envoy\") pod \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\" (UID: \"32f50b9e-f1e8-47e9-8e50-e3ba166af2de\") "
Oct 30 18:57:48.494554 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.494384    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-socket" pod "32f50b9e-f1e8-47e9-8e50-e3ba166af2de" (UID: "32f50b9e-f1e8-47e9-8e50-e3ba166af2de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.495309 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.494817    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istiod-ca-cert" pod "32f50b9e-f1e8-47e9-8e50-e3ba166af2de" (UID: "32f50b9e-f1e8-47e9-8e50-e3ba166af2de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:48.495618 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.495447    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-data" pod "32f50b9e-f1e8-47e9-8e50-e3ba166af2de" (UID: "32f50b9e-f1e8-47e9-8e50-e3ba166af2de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.497510 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.495781    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-certs" pod "32f50b9e-f1e8-47e9-8e50-e3ba166af2de" (UID: "32f50b9e-f1e8-47e9-8e50-e3ba166af2de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.497510 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.495989    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.497510 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.496539    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.498013 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.497980    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-token" pod "32f50b9e-f1e8-47e9-8e50-e3ba166af2de" (UID: "32f50b9e-f1e8-47e9-8e50-e3ba166af2de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:48.502394 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.502275    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-podinfo" pod "32f50b9e-f1e8-47e9-8e50-e3ba166af2de" (UID: "32f50b9e-f1e8-47e9-8e50-e3ba166af2de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:57:48.502829 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.502805    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-credential-socket" pod "32f50b9e-f1e8-47e9-8e50-e3ba166af2de" (UID: "32f50b9e-f1e8-47e9-8e50-e3ba166af2de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.505876 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.505194    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-envoy" pod "32f50b9e-f1e8-47e9-8e50-e3ba166af2de" (UID: "32f50b9e-f1e8-47e9-8e50-e3ba166af2de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.511797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.511770    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-kube-api-access-t9tbk" pod "32f50b9e-f1e8-47e9-8e50-e3ba166af2de" (UID: "32f50b9e-f1e8-47e9-8e50-e3ba166af2de"). InnerVolumeSpecName "kube-api-access-t9tbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:48.520395 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.520375    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:57:48.522850 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.522829    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:57:48.597435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.597398    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49765651-6ef5-47c8-aef6-03b21952a2ad-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49765651-6ef5-47c8-aef6-03b21952a2ad-istiod-ca-cert\") pod \"49765651-6ef5-47c8-aef6-03b21952a2ad\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") "
Oct 30 18:57:48.597626 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.597465    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-envoy\") pod \"1aa78826-8a83-4c64-b87e-642678bf9774\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") "
Oct 30 18:57:48.597626 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.597527    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1aa78826-8a83-4c64-b87e-642678bf9774-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa78826-8a83-4c64-b87e-642678bf9774-istiod-ca-cert\") pod \"1aa78826-8a83-4c64-b87e-642678bf9774\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") "
Oct 30 18:57:48.597626 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.597568    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-kube-api-access-bnqwp\" (UniqueName: \"kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-kube-api-access-bnqwp\") pod \"1aa78826-8a83-4c64-b87e-642678bf9774\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") "
Oct 30 18:57:48.597814 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.597626    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-credential-socket\") pod \"49765651-6ef5-47c8-aef6-03b21952a2ad\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") "
Oct 30 18:57:48.597814 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.597724    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-socket\") pod \"1aa78826-8a83-4c64-b87e-642678bf9774\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") "
Oct 30 18:57:48.597814 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.597762    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-socket\") pod \"49765651-6ef5-47c8-aef6-03b21952a2ad\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") "
Oct 30 18:57:48.597814 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.597809    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-credential-socket\") pod \"1aa78826-8a83-4c64-b87e-642678bf9774\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") "
Oct 30 18:57:48.598072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.597838    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-podinfo\") pod \"49765651-6ef5-47c8-aef6-03b21952a2ad\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") "
Oct 30 18:57:48.598072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.597879    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-certs\") pod \"49765651-6ef5-47c8-aef6-03b21952a2ad\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") "
Oct 30 18:57:48.598072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.597917    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-token\" (UniqueName: \"kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-token\") pod \"49765651-6ef5-47c8-aef6-03b21952a2ad\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") "
Oct 30 18:57:48.598072 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.597989    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-istio-token\" (UniqueName: \"kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-istio-token\") pod \"1aa78826-8a83-4c64-b87e-642678bf9774\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") "
Oct 30 18:57:48.598369 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.598343    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-socket" pod "1aa78826-8a83-4c64-b87e-642678bf9774" (UID: "1aa78826-8a83-4c64-b87e-642678bf9774"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.598693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.598668    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49765651-6ef5-47c8-aef6-03b21952a2ad-istiod-ca-cert" pod "49765651-6ef5-47c8-aef6-03b21952a2ad" (UID: "49765651-6ef5-47c8-aef6-03b21952a2ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:48.598990 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.598021    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-data\") pod \"49765651-6ef5-47c8-aef6-03b21952a2ad\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") "
Oct 30 18:57:48.599070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.599053    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-kube-api-access-mg9kr\" (UniqueName: \"kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-kube-api-access-mg9kr\") pod \"49765651-6ef5-47c8-aef6-03b21952a2ad\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") "
Oct 30 18:57:48.599197 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.599176    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-envoy\") pod \"49765651-6ef5-47c8-aef6-03b21952a2ad\" (UID: \"49765651-6ef5-47c8-aef6-03b21952a2ad\") "
Oct 30 18:57:48.599267 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.599216    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1aa78826-8a83-4c64-b87e-642678bf9774-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa78826-8a83-4c64-b87e-642678bf9774-istio-podinfo\") pod \"1aa78826-8a83-4c64-b87e-642678bf9774\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") "
Oct 30 18:57:48.599327 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.599272    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-certs\") pod \"1aa78826-8a83-4c64-b87e-642678bf9774\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") "
Oct 30 18:57:48.599498 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.599477    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-data\") pod \"1aa78826-8a83-4c64-b87e-642678bf9774\" (UID: \"1aa78826-8a83-4c64-b87e-642678bf9774\") "
Oct 30 18:57:48.600159 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.600136    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.600242 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.600163    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.600242 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.600179    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.600242 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.600193    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9tbk\" (UniqueName: \"kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-kube-api-access-t9tbk\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.600242 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.600208    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.600242 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.600222    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.600242 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.600236    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.600564 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.600250    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32f50b9e-f1e8-47e9-8e50-e3ba166af2de-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.600564 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.600265    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49765651-6ef5-47c8-aef6-03b21952a2ad-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.600682 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.600640    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-certs" pod "49765651-6ef5-47c8-aef6-03b21952a2ad" (UID: "49765651-6ef5-47c8-aef6-03b21952a2ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.601256 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.601230    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-credential-socket" pod "1aa78826-8a83-4c64-b87e-642678bf9774" (UID: "1aa78826-8a83-4c64-b87e-642678bf9774"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.601442 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.601421    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-certs" pod "1aa78826-8a83-4c64-b87e-642678bf9774" (UID: "1aa78826-8a83-4c64-b87e-642678bf9774"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.602925 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.602895    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-socket" pod "49765651-6ef5-47c8-aef6-03b21952a2ad" (UID: "49765651-6ef5-47c8-aef6-03b21952a2ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.603038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.602950    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-data" pod "49765651-6ef5-47c8-aef6-03b21952a2ad" (UID: "49765651-6ef5-47c8-aef6-03b21952a2ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.603205 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.603183    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-envoy" pod "1aa78826-8a83-4c64-b87e-642678bf9774" (UID: "1aa78826-8a83-4c64-b87e-642678bf9774"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.603371 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.603355    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-credential-socket" pod "49765651-6ef5-47c8-aef6-03b21952a2ad" (UID: "49765651-6ef5-47c8-aef6-03b21952a2ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.603499 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.603485    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-data" pod "1aa78826-8a83-4c64-b87e-642678bf9774" (UID: "1aa78826-8a83-4c64-b87e-642678bf9774"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.603566 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.603483    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-token" pod "49765651-6ef5-47c8-aef6-03b21952a2ad" (UID: "49765651-6ef5-47c8-aef6-03b21952a2ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:48.603718 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.603700    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aa78826-8a83-4c64-b87e-642678bf9774-istiod-ca-cert" pod "1aa78826-8a83-4c64-b87e-642678bf9774" (UID: "1aa78826-8a83-4c64-b87e-642678bf9774"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:48.604044 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.604016    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-kube-api-access-bnqwp" pod "1aa78826-8a83-4c64-b87e-642678bf9774" (UID: "1aa78826-8a83-4c64-b87e-642678bf9774"). InnerVolumeSpecName "kube-api-access-bnqwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:48.604626 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.604601    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1aa78826-8a83-4c64-b87e-642678bf9774-istio-podinfo" pod "1aa78826-8a83-4c64-b87e-642678bf9774" (UID: "1aa78826-8a83-4c64-b87e-642678bf9774"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:57:48.606338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.606309    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-podinfo" pod "49765651-6ef5-47c8-aef6-03b21952a2ad" (UID: "49765651-6ef5-47c8-aef6-03b21952a2ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:57:48.607532 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.607066    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-envoy" pod "49765651-6ef5-47c8-aef6-03b21952a2ad" (UID: "49765651-6ef5-47c8-aef6-03b21952a2ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:57:48.607532 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.607460    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-kube-api-access-mg9kr" pod "49765651-6ef5-47c8-aef6-03b21952a2ad" (UID: "49765651-6ef5-47c8-aef6-03b21952a2ad"). InnerVolumeSpecName "kube-api-access-mg9kr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:48.607663 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.607640    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-istio-token" pod "1aa78826-8a83-4c64-b87e-642678bf9774" (UID: "1aa78826-8a83-4c64-b87e-642678bf9774"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:48.701172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701139    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1aa78826-8a83-4c64-b87e-642678bf9774-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701174    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnqwp\" (UniqueName: \"kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-kube-api-access-bnqwp\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701186    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701194    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701204    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701213    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701222    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701231    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701239    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1aa78826-8a83-4c64-b87e-642678bf9774-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701247    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701255    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg9kr\" (UniqueName: \"kubernetes.io/projected/49765651-6ef5-47c8-aef6-03b21952a2ad-kube-api-access-mg9kr\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701264    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49765651-6ef5-47c8-aef6-03b21952a2ad-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701272    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1aa78826-8a83-4c64-b87e-642678bf9774-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701280    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701288    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.701391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.701295    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1aa78826-8a83-4c64-b87e-642678bf9774-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:48.745409 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:48.745376    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:49.047121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.036629    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:49.184481 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.184383    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p" podUID="4b33bf67-2556-4670-9fc9-fbe0e5e52877" containerName="payload-processing" containerID="cri-o://7a52a93636461e83f68b1df616e634c682682e2eca06bf08a51c088817a193c9" gracePeriod=25
Oct 30 18:57:49.390475 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.389687    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"
Oct 30 18:57:49.390475 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.389672    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2" event={"ID":"32f50b9e-f1e8-47e9-8e50-e3ba166af2de","Type":"ContainerDied","Data":"c6fe51cb36699c163453bdabbcfec430f3f3b3fc4c6969c4cccf709a6b7df3a5"}
Oct 30 18:57:49.390475 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.389838    4925 scope.go:122] "RemoveContainer" containerID="bf4ec6f99c51191b869e5fa80d3d6a2529efcea12f0da827ba3cf0082c2e8c3f"
Oct 30 18:57:49.392868 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.392837    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"
Oct 30 18:57:49.392982 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.392925    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x" event={"ID":"1aa78826-8a83-4c64-b87e-642678bf9774","Type":"ContainerDied","Data":"0607a5d20066171e9490b29a2edc55037cd3adb3b3c0be0abadd732d04fd2a22"}
Oct 30 18:57:49.404332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.395329    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g" event={"ID":"49765651-6ef5-47c8-aef6-03b21952a2ad","Type":"ContainerDied","Data":"dc7e53e742c632f073f15a30ba34db8d053ed3f90a5ccc5c6334dc509904d6e7"}
Oct 30 18:57:49.404332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.395438    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"
Oct 30 18:57:49.404332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.402943    4925 generic.go:356] "Generic (PLEG): container finished" podID="4b33bf67-2556-4670-9fc9-fbe0e5e52877" containerID="7a52a93636461e83f68b1df616e634c682682e2eca06bf08a51c088817a193c9" exitCode=0
Oct 30 18:57:49.404332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.403796    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p" event={"ID":"4b33bf67-2556-4670-9fc9-fbe0e5e52877","Type":"ContainerDied","Data":"7a52a93636461e83f68b1df616e634c682682e2eca06bf08a51c088817a193c9"}
Oct 30 18:57:49.415493 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.405875    4925 scope.go:122] "RemoveContainer" containerID="f6fdc76ecc10a2b164858fd8a6e0509acff9b518212ab8c4a3671a5a287f3f7e"
Oct 30 18:57:49.431420 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.431400    4925 scope.go:122] "RemoveContainer" containerID="0ca7a4a6c2d2208f156e85652e4248cb2749b8a70f781d2fd3727804df5e3cce"
Oct 30 18:57:49.432065 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.432045    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"]
Oct 30 18:57:49.439358 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.438817    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-jrf7x"]
Oct 30 18:57:49.455239 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.455142    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"]
Oct 30 18:57:49.459112 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.458989    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-z655g"]
Oct 30 18:57:49.480190 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.480165    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"]
Oct 30 18:57:49.487474 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.487452    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-4f8j2"]
Oct 30 18:57:49.508958 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.508934    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"
Oct 30 18:57:49.550629 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.550598    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b33bf67-2556-4670-9fc9-fbe0e5e52877-kube-api-access-86fcq\" (UniqueName: \"kubernetes.io/projected/4b33bf67-2556-4670-9fc9-fbe0e5e52877-kube-api-access-86fcq\") pod \"4b33bf67-2556-4670-9fc9-fbe0e5e52877\" (UID: \"4b33bf67-2556-4670-9fc9-fbe0e5e52877\") "
Oct 30 18:57:49.550759 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.550678    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b33bf67-2556-4670-9fc9-fbe0e5e52877-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b33bf67-2556-4670-9fc9-fbe0e5e52877-plugins-config-volume\") pod \"4b33bf67-2556-4670-9fc9-fbe0e5e52877\" (UID: \"4b33bf67-2556-4670-9fc9-fbe0e5e52877\") "
Oct 30 18:57:49.551275 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.551244    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b33bf67-2556-4670-9fc9-fbe0e5e52877-plugins-config-volume" pod "4b33bf67-2556-4670-9fc9-fbe0e5e52877" (UID: "4b33bf67-2556-4670-9fc9-fbe0e5e52877"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:49.551842 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.551820    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b33bf67-2556-4670-9fc9-fbe0e5e52877-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:49.555040 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.555013    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b33bf67-2556-4670-9fc9-fbe0e5e52877-kube-api-access-86fcq" pod "4b33bf67-2556-4670-9fc9-fbe0e5e52877" (UID: "4b33bf67-2556-4670-9fc9-fbe0e5e52877"). InnerVolumeSpecName "kube-api-access-86fcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:49.653527 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.653497    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86fcq\" (UniqueName: \"kubernetes.io/projected/4b33bf67-2556-4670-9fc9-fbe0e5e52877-kube-api-access-86fcq\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:49.997510 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:49.997447    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n" podUID="01d7f5f2-df5b-44c0-a81b-3fe176212a1f" containerName="payload-processing" containerID="cri-o://9eb44fbbf2bfc5bd7d5471e7f040357d0c6581b786dd11666ce9dd9e14cd1109" gracePeriod=25
Oct 30 18:57:50.101048 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.100994    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:50.118390 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.118363    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"]
Oct 30 18:57:50.119335 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119311    4925 cpu_manager.go:414] "containerMap: removing container" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy"
Oct 30 18:57:50.119335 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119334    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy"
Oct 30 18:57:50.119531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119380    4925 cpu_manager.go:414] "containerMap: removing container" podUID="4b33bf67-2556-4670-9fc9-fbe0e5e52877" containerName="payload-processing"
Oct 30 18:57:50.119531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119390    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b33bf67-2556-4670-9fc9-fbe0e5e52877" containerName="payload-processing"
Oct 30 18:57:50.119531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119404    4925 cpu_manager.go:414] "containerMap: removing container" podUID="1aa78826-8a83-4c64-b87e-642678bf9774" containerName="istio-proxy"
Oct 30 18:57:50.119531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119411    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1aa78826-8a83-4c64-b87e-642678bf9774" containerName="istio-proxy"
Oct 30 18:57:50.119531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119428    4925 cpu_manager.go:414] "containerMap: removing container" podUID="32f50b9e-f1e8-47e9-8e50-e3ba166af2de" containerName="istio-proxy"
Oct 30 18:57:50.119531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119434    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32f50b9e-f1e8-47e9-8e50-e3ba166af2de" containerName="istio-proxy"
Oct 30 18:57:50.119531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119455    4925 cpu_manager.go:414] "containerMap: removing container" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy"
Oct 30 18:57:50.119531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119462    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy"
Oct 30 18:57:50.119926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119648    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy"
Oct 30 18:57:50.119926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119671    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b33bf67-2556-4670-9fc9-fbe0e5e52877" containerName="payload-processing"
Oct 30 18:57:50.119926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119681    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="1aa78826-8a83-4c64-b87e-642678bf9774" containerName="istio-proxy"
Oct 30 18:57:50.119926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119694    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="32f50b9e-f1e8-47e9-8e50-e3ba166af2de" containerName="istio-proxy"
Oct 30 18:57:50.119926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.119708    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" containerName="istio-proxy"
Oct 30 18:57:50.123654 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.123633    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"]
Oct 30 18:57:50.123846 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.123824    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"
Oct 30 18:57:50.129889 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.129868    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"
Oct 30 18:57:50.133486 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.133461    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-6a327a\""
Oct 30 18:57:50.133571 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.133509    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"]
Oct 30 18:57:50.137933 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.137909    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"]
Oct 30 18:57:50.143304 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.140575    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-6a327a-dockercfg-tj97l\""
Oct 30 18:57:50.180115 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.179241    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd" podUID="bdc97dc0-498f-419e-bfd1-072b72fb8658" containerName="payload-pre-processing" containerID="cri-o://a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9" gracePeriod=25
Oct 30 18:57:50.309171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.303208    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w\" (UID: \"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"
Oct 30 18:57:50.309171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.303278    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4rn5\" (UniqueName: \"kubernetes.io/projected/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-kube-api-access-f4rn5\") pod \"payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w\" (UID: \"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"
Oct 30 18:57:50.309171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.303331    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b92c2\" (UniqueName: \"kubernetes.io/projected/3823fc1b-a60f-4315-ae97-db1bc2e6974b-kube-api-access-b92c2\") pod \"payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t\" (UID: \"3823fc1b-a60f-4315-ae97-db1bc2e6974b\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"
Oct 30 18:57:50.309171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.303530    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3823fc1b-a60f-4315-ae97-db1bc2e6974b-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t\" (UID: \"3823fc1b-a60f-4315-ae97-db1bc2e6974b\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"
Oct 30 18:57:50.448124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.434654    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:50.448124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.440834    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3823fc1b-a60f-4315-ae97-db1bc2e6974b-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t\" (UID: \"3823fc1b-a60f-4315-ae97-db1bc2e6974b\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"
Oct 30 18:57:50.448124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.443317    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w\" (UID: \"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"
Oct 30 18:57:50.448124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.443394    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4rn5\" (UniqueName: \"kubernetes.io/projected/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-kube-api-access-f4rn5\") pod \"payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w\" (UID: \"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"
Oct 30 18:57:50.448124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.443445    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b92c2\" (UniqueName: \"kubernetes.io/projected/3823fc1b-a60f-4315-ae97-db1bc2e6974b-kube-api-access-b92c2\") pod \"payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t\" (UID: \"3823fc1b-a60f-4315-ae97-db1bc2e6974b\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"
Oct 30 18:57:50.448124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.445279    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3823fc1b-a60f-4315-ae97-db1bc2e6974b-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t\" (UID: \"3823fc1b-a60f-4315-ae97-db1bc2e6974b\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"
Oct 30 18:57:50.448124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.446066    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w\" (UID: \"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"
Oct 30 18:57:50.515260 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.499677    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b92c2\" (UniqueName: \"kubernetes.io/projected/3823fc1b-a60f-4315-ae97-db1bc2e6974b-kube-api-access-b92c2\") pod \"payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t\" (UID: \"3823fc1b-a60f-4315-ae97-db1bc2e6974b\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"
Oct 30 18:57:50.569687 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.569636    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4rn5\" (UniqueName: \"kubernetes.io/projected/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-kube-api-access-f4rn5\") pod \"payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w\" (UID: \"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"
Oct 30 18:57:50.570789 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.570756    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1aa78826-8a83-4c64-b87e-642678bf9774" path="/var/lib/kubelet/pods/1aa78826-8a83-4c64-b87e-642678bf9774/volumes"
Oct 30 18:57:50.571578 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.571551    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32f50b9e-f1e8-47e9-8e50-e3ba166af2de" path="/var/lib/kubelet/pods/32f50b9e-f1e8-47e9-8e50-e3ba166af2de/volumes"
Oct 30 18:57:50.572900 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.572226    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49765651-6ef5-47c8-aef6-03b21952a2ad" path="/var/lib/kubelet/pods/49765651-6ef5-47c8-aef6-03b21952a2ad/volumes"
Oct 30 18:57:50.574585 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.574559    4925 generic.go:356] "Generic (PLEG): container finished" podID="01d7f5f2-df5b-44c0-a81b-3fe176212a1f" containerID="9eb44fbbf2bfc5bd7d5471e7f040357d0c6581b786dd11666ce9dd9e14cd1109" exitCode=0
Oct 30 18:57:50.574755 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.574732    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n" event={"ID":"01d7f5f2-df5b-44c0-a81b-3fe176212a1f","Type":"ContainerDied","Data":"9eb44fbbf2bfc5bd7d5471e7f040357d0c6581b786dd11666ce9dd9e14cd1109"}
Oct 30 18:57:50.578675 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.578648    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p" event={"ID":"4b33bf67-2556-4670-9fc9-fbe0e5e52877","Type":"ContainerDied","Data":"73e62dceefa099f2403207ab4564b46b6be7d72c86d2f8de520de583cfbf99bf"}
Oct 30 18:57:50.578769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.578711    4925 scope.go:122] "RemoveContainer" containerID="7a52a93636461e83f68b1df616e634c682682e2eca06bf08a51c088817a193c9"
Oct 30 18:57:50.578854 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.578839    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"
Oct 30 18:57:50.640026 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.639994    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"
Oct 30 18:57:50.655227 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.655161    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"]
Oct 30 18:57:50.660019 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.659994    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-12a6e1-698647d9b-rwr4p"]
Oct 30 18:57:50.675960 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.675837    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-kube-api-access-hqrbd\" (UniqueName: \"kubernetes.io/projected/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-kube-api-access-hqrbd\") pod \"01d7f5f2-df5b-44c0-a81b-3fe176212a1f\" (UID: \"01d7f5f2-df5b-44c0-a81b-3fe176212a1f\") "
Oct 30 18:57:50.676199 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.676064    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-plugins-config-volume\") pod \"01d7f5f2-df5b-44c0-a81b-3fe176212a1f\" (UID: \"01d7f5f2-df5b-44c0-a81b-3fe176212a1f\") "
Oct 30 18:57:50.677271 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.676983    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-plugins-config-volume" pod "01d7f5f2-df5b-44c0-a81b-3fe176212a1f" (UID: "01d7f5f2-df5b-44c0-a81b-3fe176212a1f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:50.684648 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.684540    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-kube-api-access-hqrbd" pod "01d7f5f2-df5b-44c0-a81b-3fe176212a1f" (UID: "01d7f5f2-df5b-44c0-a81b-3fe176212a1f"). InnerVolumeSpecName "kube-api-access-hqrbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:50.754363 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.754329    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"
Oct 30 18:57:50.772525 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.772499    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"
Oct 30 18:57:50.778404 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.778380    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"
Oct 30 18:57:50.779693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.779670    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:50.779794 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.779698    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqrbd\" (UniqueName: \"kubernetes.io/projected/01d7f5f2-df5b-44c0-a81b-3fe176212a1f-kube-api-access-hqrbd\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:50.790993 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.790966    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"]
Oct 30 18:57:50.792030 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.791996    4925 cpu_manager.go:414] "containerMap: removing container" podUID="bdc97dc0-498f-419e-bfd1-072b72fb8658" containerName="payload-pre-processing"
Oct 30 18:57:50.792030 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.792020    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdc97dc0-498f-419e-bfd1-072b72fb8658" containerName="payload-pre-processing"
Oct 30 18:57:50.792286 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.792061    4925 cpu_manager.go:414] "containerMap: removing container" podUID="01d7f5f2-df5b-44c0-a81b-3fe176212a1f" containerName="payload-processing"
Oct 30 18:57:50.792286 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.792069    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01d7f5f2-df5b-44c0-a81b-3fe176212a1f" containerName="payload-processing"
Oct 30 18:57:50.792286 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.792278    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdc97dc0-498f-419e-bfd1-072b72fb8658" containerName="payload-pre-processing"
Oct 30 18:57:50.792492 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.792298    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="01d7f5f2-df5b-44c0-a81b-3fe176212a1f" containerName="payload-processing"
Oct 30 18:57:50.801454 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.801428    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:50.810398 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.810321    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"]
Oct 30 18:57:50.810512 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.810439    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"
Oct 30 18:57:50.815266 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.815209    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"]
Oct 30 18:57:50.816365 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.816308    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-54bc2b-dockercfg-5ssgq\""
Oct 30 18:57:50.838181 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.838153    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"
Oct 30 18:57:50.849898 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.849865    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"]
Oct 30 18:57:50.881067 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.881036    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bdc97dc0-498f-419e-bfd1-072b72fb8658-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc97dc0-498f-419e-bfd1-072b72fb8658-plugins-config-volume\") pod \"bdc97dc0-498f-419e-bfd1-072b72fb8658\" (UID: \"bdc97dc0-498f-419e-bfd1-072b72fb8658\") "
Oct 30 18:57:50.881185 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.881128    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdc97dc0-498f-419e-bfd1-072b72fb8658-kube-api-access-7gnns\" (UniqueName: \"kubernetes.io/projected/bdc97dc0-498f-419e-bfd1-072b72fb8658-kube-api-access-7gnns\") pod \"bdc97dc0-498f-419e-bfd1-072b72fb8658\" (UID: \"bdc97dc0-498f-419e-bfd1-072b72fb8658\") "
Oct 30 18:57:50.881548 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.881518    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr\" (UID: \"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"
Oct 30 18:57:50.881817 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.881797    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f6a689f-9e21-473c-b918-e2a05e18c87f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5\" (UID: \"0f6a689f-9e21-473c-b918-e2a05e18c87f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"
Oct 30 18:57:50.882385 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.882359    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvdhl\" (UniqueName: \"kubernetes.io/projected/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-kube-api-access-wvdhl\") pod \"payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr\" (UID: \"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"
Oct 30 18:57:50.882544 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.882524    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blk2l\" (UniqueName: \"kubernetes.io/projected/0f6a689f-9e21-473c-b918-e2a05e18c87f-kube-api-access-blk2l\") pod \"payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5\" (UID: \"0f6a689f-9e21-473c-b918-e2a05e18c87f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"
Oct 30 18:57:50.883030 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.883006    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdc97dc0-498f-419e-bfd1-072b72fb8658-plugins-config-volume" pod "bdc97dc0-498f-419e-bfd1-072b72fb8658" (UID: "bdc97dc0-498f-419e-bfd1-072b72fb8658"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:50.884875 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:50.884832    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdc97dc0-498f-419e-bfd1-072b72fb8658-kube-api-access-7gnns" pod "bdc97dc0-498f-419e-bfd1-072b72fb8658" (UID: "bdc97dc0-498f-419e-bfd1-072b72fb8658"). InnerVolumeSpecName "kube-api-access-7gnns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:51.108105 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.066745    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:51.108105 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.068731    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr\" (UID: \"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"
Oct 30 18:57:51.108105 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.068777    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f6a689f-9e21-473c-b918-e2a05e18c87f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5\" (UID: \"0f6a689f-9e21-473c-b918-e2a05e18c87f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"
Oct 30 18:57:51.108105 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.068921    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvdhl\" (UniqueName: \"kubernetes.io/projected/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-kube-api-access-wvdhl\") pod \"payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr\" (UID: \"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"
Oct 30 18:57:51.108105 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.069026    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blk2l\" (UniqueName: \"kubernetes.io/projected/0f6a689f-9e21-473c-b918-e2a05e18c87f-kube-api-access-blk2l\") pod \"payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5\" (UID: \"0f6a689f-9e21-473c-b918-e2a05e18c87f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"
Oct 30 18:57:51.108105 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.069196    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc97dc0-498f-419e-bfd1-072b72fb8658-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:51.108105 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.069217    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gnns\" (UniqueName: \"kubernetes.io/projected/bdc97dc0-498f-419e-bfd1-072b72fb8658-kube-api-access-7gnns\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:57:51.108105 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.072206    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f6a689f-9e21-473c-b918-e2a05e18c87f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5\" (UID: \"0f6a689f-9e21-473c-b918-e2a05e18c87f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"
Oct 30 18:57:51.108105 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.087475    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr\" (UID: \"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"
Oct 30 18:57:51.134958 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.134920    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvdhl\" (UniqueName: \"kubernetes.io/projected/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-kube-api-access-wvdhl\") pod \"payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr\" (UID: \"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"
Oct 30 18:57:51.182625 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.182329    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"
Oct 30 18:57:51.190422 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.189660    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blk2l\" (UniqueName: \"kubernetes.io/projected/0f6a689f-9e21-473c-b918-e2a05e18c87f-kube-api-access-blk2l\") pod \"payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5\" (UID: \"0f6a689f-9e21-473c-b918-e2a05e18c87f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"
Oct 30 18:57:51.291605 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:57:51.291566    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3823fc1b_a60f_4315_ae97_db1bc2e6974b.slice/crio-abbcd3c482b66b6e39ae89936d6a1c0b025a1d6eced1cf0a15867ed055927966 WatchSource:0}: Error finding container abbcd3c482b66b6e39ae89936d6a1c0b025a1d6eced1cf0a15867ed055927966: Status 404 returned error can't find the container with id abbcd3c482b66b6e39ae89936d6a1c0b025a1d6eced1cf0a15867ed055927966
Oct 30 18:57:51.296861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.296040    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"]
Oct 30 18:57:51.343977 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.343951    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"]
Oct 30 18:57:51.346551 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:57:51.346522    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b86ce5f_c95f_4ad6_a785_71c8a5e6aaa6.slice/crio-3c7422fefc531696c8f0a77d4735dd675646173f580af64971811fba435b51c2 WatchSource:0}: Error finding container 3c7422fefc531696c8f0a77d4735dd675646173f580af64971811fba435b51c2: Status 404 returned error can't find the container with id 3c7422fefc531696c8f0a77d4735dd675646173f580af64971811fba435b51c2
Oct 30 18:57:51.455200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.455174    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"
Oct 30 18:57:51.555733 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.555645    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"]
Oct 30 18:57:51.601790 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.588440    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:51.654652 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.654616    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n" event={"ID":"01d7f5f2-df5b-44c0-a81b-3fe176212a1f","Type":"ContainerDied","Data":"9f9dc91c9b6d03c9421b79ec5e3fd6de4a94791d40f5a8f02ea5f7070a4d2234"}
Oct 30 18:57:51.654769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.654688    4925 scope.go:122] "RemoveContainer" containerID="9eb44fbbf2bfc5bd7d5471e7f040357d0c6581b786dd11666ce9dd9e14cd1109"
Oct 30 18:57:51.654855 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.654840    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"
Oct 30 18:57:51.673197 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.666784    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t" event={"ID":"3823fc1b-a60f-4315-ae97-db1bc2e6974b","Type":"ContainerStarted","Data":"5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e"}
Oct 30 18:57:51.673197 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.666836    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t" event={"ID":"3823fc1b-a60f-4315-ae97-db1bc2e6974b","Type":"ContainerStarted","Data":"abbcd3c482b66b6e39ae89936d6a1c0b025a1d6eced1cf0a15867ed055927966"}
Oct 30 18:57:51.673197 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.668065    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"
Oct 30 18:57:51.673197 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.672544    4925 generic.go:356] "Generic (PLEG): container finished" podID="bdc97dc0-498f-419e-bfd1-072b72fb8658" containerID="a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9" exitCode=0
Oct 30 18:57:51.673197 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.672626    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd" event={"ID":"bdc97dc0-498f-419e-bfd1-072b72fb8658","Type":"ContainerDied","Data":"a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9"}
Oct 30 18:57:51.673197 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.672669    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd" event={"ID":"bdc97dc0-498f-419e-bfd1-072b72fb8658","Type":"ContainerDied","Data":"74fd1cefbd15c6183ba83cc70f8b8d3fd17cd126b91a0e243fd48ea93eb7f24d"}
Oct 30 18:57:51.673197 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.672755    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"
Oct 30 18:57:51.677148 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.677126    4925 scope.go:122] "RemoveContainer" containerID="a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9"
Oct 30 18:57:51.681238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.681212    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w" event={"ID":"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6","Type":"ContainerStarted","Data":"1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb"}
Oct 30 18:57:51.681315 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.681260    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w" event={"ID":"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6","Type":"ContainerStarted","Data":"3c7422fefc531696c8f0a77d4735dd675646173f580af64971811fba435b51c2"}
Oct 30 18:57:51.688184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.681413    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"
Oct 30 18:57:51.697011 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.696718    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t" podStartSLOduration=1.696702074 podStartE2EDuration="1.696702074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:51.696367587 +0000 UTC m=+1293.412712664" watchObservedRunningTime="2026-10-30 18:57:51.696702074 +0000 UTC m=+1293.413047152"
Oct 30 18:57:51.704705 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.704683    4925 scope.go:122] "RemoveContainer" containerID="a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9"
Oct 30 18:57:51.705427 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:57:51.705363    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9\": container with ID starting with a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9 not found: ID does not exist" containerID="a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9"
Oct 30 18:57:51.705592 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.705559    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9"} err="failed to get container status \"a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9\": rpc error: code = NotFound desc = could not find container \"a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9\": container with ID starting with a266dd1d720aa54707b201ea3750b0fbee19d0123c86094c07ee443327de88c9 not found: ID does not exist"
Oct 30 18:57:51.712436 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.712394    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w" podStartSLOduration=1.712377917 podStartE2EDuration="1.712377917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:51.710903236 +0000 UTC m=+1293.427248314" watchObservedRunningTime="2026-10-30 18:57:51.712377917 +0000 UTC m=+1293.428722994"
Oct 30 18:57:51.726873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.726851    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"]
Oct 30 18:57:51.730280 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.730262    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-5fd7f74668-5zd2n"]
Oct 30 18:57:51.745129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.745089    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"]
Oct 30 18:57:51.750877 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.750856    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-585958d99f-j6dbd"]
Oct 30 18:57:51.772350 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:51.772313    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"]
Oct 30 18:57:51.775399 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:57:51.775044    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f6a689f_9e21_473c_b918_e2a05e18c87f.slice/crio-51c038ff09d9adc393e2957787217df0e52a24162e5b65af23832ecf521ff146 WatchSource:0}: Error finding container 51c038ff09d9adc393e2957787217df0e52a24162e5b65af23832ecf521ff146: Status 404 returned error can't find the container with id 51c038ff09d9adc393e2957787217df0e52a24162e5b65af23832ecf521ff146
Oct 30 18:57:52.488124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:52.474866    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:52.488124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:52.482873    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01d7f5f2-df5b-44c0-a81b-3fe176212a1f" path="/var/lib/kubelet/pods/01d7f5f2-df5b-44c0-a81b-3fe176212a1f/volumes"
Oct 30 18:57:52.488124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:52.483484    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b33bf67-2556-4670-9fc9-fbe0e5e52877" path="/var/lib/kubelet/pods/4b33bf67-2556-4670-9fc9-fbe0e5e52877/volumes"
Oct 30 18:57:52.488124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:52.483963    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdc97dc0-498f-419e-bfd1-072b72fb8658" path="/var/lib/kubelet/pods/bdc97dc0-498f-419e-bfd1-072b72fb8658/volumes"
Oct 30 18:57:52.694070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:52.693757    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr" event={"ID":"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4","Type":"ContainerStarted","Data":"ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a"}
Oct 30 18:57:52.694070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:52.693841    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"
Oct 30 18:57:52.694070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:52.693996    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr" event={"ID":"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4","Type":"ContainerStarted","Data":"88d409184d372b77cafec0efee5a2d6a2df4d669d648eb203dfaa815403d601e"}
Oct 30 18:57:52.701133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:52.698045    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5" event={"ID":"0f6a689f-9e21-473c-b918-e2a05e18c87f","Type":"ContainerStarted","Data":"566be32b80ab735f73bda0a5642f8f85c99d677c13afb6d0898c8911c666b199"}
Oct 30 18:57:52.701133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:52.698118    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"
Oct 30 18:57:52.701133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:52.698143    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5" event={"ID":"0f6a689f-9e21-473c-b918-e2a05e18c87f","Type":"ContainerStarted","Data":"51c038ff09d9adc393e2957787217df0e52a24162e5b65af23832ecf521ff146"}
Oct 30 18:57:52.712038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:52.711985    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr" podStartSLOduration=2.711968631 podStartE2EDuration="2.711968631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:52.709644566 +0000 UTC m=+1294.425989660" watchObservedRunningTime="2026-10-30 18:57:52.711968631 +0000 UTC m=+1294.428313709"
Oct 30 18:57:52.729327 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:52.729270    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5" podStartSLOduration=2.7292557889999998 podStartE2EDuration="2.729255789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:52.724753996 +0000 UTC m=+1294.441099072" watchObservedRunningTime="2026-10-30 18:57:52.729255789 +0000 UTC m=+1294.445600864"
Oct 30 18:57:53.269541 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:53.269505    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:53.869136 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:53.869080    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:54.585152 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:54.580713    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:55.259902 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:55.259870    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:55.993403 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:55.993348    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:56.514518 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.514485    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"]
Oct 30 18:57:56.521174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.521083    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.528242 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.528196    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-6a327a-serving-cert\""
Oct 30 18:57:56.530348 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.530325    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"]
Oct 30 18:57:56.630948 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.630892    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-maas-api-tls\") pod \"maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.631180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.630957    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-metrics-certs\") pod \"maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.631180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.631056    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80192118-9d8a-4340-b466-e4f24a1bb08f-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.631297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.631236    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64k9d\" (UniqueName: \"kubernetes.io/projected/80192118-9d8a-4340-b466-e4f24a1bb08f-kube-api-access-64k9d\") pod \"maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.706123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.705319    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:56.734119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.734059    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64k9d\" (UniqueName: \"kubernetes.io/projected/80192118-9d8a-4340-b466-e4f24a1bb08f-kube-api-access-64k9d\") pod \"maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.734295 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.734273    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-maas-api-tls\") pod \"maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.734350 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.734314    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-metrics-certs\") pod \"maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.734420 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.734406    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80192118-9d8a-4340-b466-e4f24a1bb08f-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.735421 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.735392    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80192118-9d8a-4340-b466-e4f24a1bb08f-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.756504 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.753656    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-maas-api-tls\") pod \"maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.759982 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.759903    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64k9d\" (UniqueName: \"kubernetes.io/projected/80192118-9d8a-4340-b466-e4f24a1bb08f-kube-api-access-64k9d\") pod \"maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.767717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.767659    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-metrics-certs\") pod \"maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:56.846258 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:56.846228    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:57.003815 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:57.003781    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:57.135055 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:57.135020    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"]
Oct 30 18:57:57.135318 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:57:57.135039    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80192118_9d8a_4340_b466_e4f24a1bb08f.slice/crio-971bf1e2182c4223a16f4a673432e7655f33d4d71e1f19e48416821de6163745 WatchSource:0}: Error finding container 971bf1e2182c4223a16f4a673432e7655f33d4d71e1f19e48416821de6163745: Status 404 returned error can't find the container with id 971bf1e2182c4223a16f4a673432e7655f33d4d71e1f19e48416821de6163745
Oct 30 18:57:57.530210 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:57.530176    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:57.751029 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:57.750997    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9" event={"ID":"80192118-9d8a-4340-b466-e4f24a1bb08f","Type":"ContainerStarted","Data":"06d2f0ded9a3e58241a056c3d41a423628376e39f5d29303ca8accbc6d49df89"}
Oct 30 18:57:57.751200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:57.751056    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9" event={"ID":"80192118-9d8a-4340-b466-e4f24a1bb08f","Type":"ContainerStarted","Data":"971bf1e2182c4223a16f4a673432e7655f33d4d71e1f19e48416821de6163745"}
Oct 30 18:57:57.751200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:57.751136    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:57:57.771225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:57.771181    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9" podStartSLOduration=1.3123741070000001 podStartE2EDuration="1.771162955s" totalImagesPullingTime="458.788848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:57.768808688 +0000 UTC m=+1299.485153777" watchObservedRunningTime="2026-10-30 18:57:57.771162955 +0000 UTC m=+1299.487508028"
Oct 30 18:57:58.292120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:58.292060    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:59.067769 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.067722    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:59.536871 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.536832    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"]
Oct 30 18:57:59.541432 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.537114    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7" podUID="db5672e5-8c89-43f8-bb92-82917a137bbd" containerName="maas-api" containerID="cri-o://be4deb66846b987a9ed4b4c58951f13bb05392d97b14aad8b9dfe0f9902d9468" gracePeriod=30
Oct 30 18:57:59.593692 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.593654    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"]
Oct 30 18:57:59.594122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.593935    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg" podUID="0876bd1c-4d76-4118-9a9a-92c43398d4a5" containerName="payload-processing" containerID="cri-o://5dc82032fc407faaf5f7630baecfc49af458681a432fcaee4691b02ed4793f52" gracePeriod=30
Oct 30 18:57:59.607133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.607079    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"]
Oct 30 18:57:59.607400 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.607367    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk" podUID="eaa10c34-997b-4362-92f3-e29f168e1659" containerName="payload-pre-processing" containerID="cri-o://a607dd7582907f9cebb76ee20ccf394190022865e4c940599da5ccf5e6779acd" gracePeriod=30
Oct 30 18:57:59.673155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.668156    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:59.779560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.777292    4925 generic.go:356] "Generic (PLEG): container finished" podID="0876bd1c-4d76-4118-9a9a-92c43398d4a5" containerID="5dc82032fc407faaf5f7630baecfc49af458681a432fcaee4691b02ed4793f52" exitCode=0
Oct 30 18:57:59.779560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.777403    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg" event={"ID":"0876bd1c-4d76-4118-9a9a-92c43398d4a5","Type":"ContainerDied","Data":"5dc82032fc407faaf5f7630baecfc49af458681a432fcaee4691b02ed4793f52"}
Oct 30 18:57:59.779560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.779332    4925 generic.go:356] "Generic (PLEG): container finished" podID="db5672e5-8c89-43f8-bb92-82917a137bbd" containerID="be4deb66846b987a9ed4b4c58951f13bb05392d97b14aad8b9dfe0f9902d9468" exitCode=0
Oct 30 18:57:59.779560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.779408    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7" event={"ID":"db5672e5-8c89-43f8-bb92-82917a137bbd","Type":"ContainerDied","Data":"be4deb66846b987a9ed4b4c58951f13bb05392d97b14aad8b9dfe0f9902d9468"}
Oct 30 18:57:59.802175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.799380    4925 generic.go:356] "Generic (PLEG): container finished" podID="eaa10c34-997b-4362-92f3-e29f168e1659" containerID="a607dd7582907f9cebb76ee20ccf394190022865e4c940599da5ccf5e6779acd" exitCode=0
Oct 30 18:57:59.802175 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.799551    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk" event={"ID":"eaa10c34-997b-4362-92f3-e29f168e1659","Type":"ContainerDied","Data":"a607dd7582907f9cebb76ee20ccf394190022865e4c940599da5ccf5e6779acd"}
Oct 30 18:57:59.831120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.829910    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7" podUID="db5672e5-8c89-43f8-bb92-82917a137bbd" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.121:8443/health\": dial tcp 10.132.0.121:8443: connect: connection refused"
Oct 30 18:57:59.848853 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:57:59.848742    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:00.168056 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.163956    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:00.312066 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.311498    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:58:00.318306 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.317736    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"
Oct 30 18:58:00.407520 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.407432    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:00.481172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.480287    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db5672e5-8c89-43f8-bb92-82917a137bbd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db5672e5-8c89-43f8-bb92-82917a137bbd-openshift-service-ca\") pod \"db5672e5-8c89-43f8-bb92-82917a137bbd\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") "
Oct 30 18:58:00.481172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.480368    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eaa10c34-997b-4362-92f3-e29f168e1659-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaa10c34-997b-4362-92f3-e29f168e1659-plugins-config-volume\") pod \"eaa10c34-997b-4362-92f3-e29f168e1659\" (UID: \"eaa10c34-997b-4362-92f3-e29f168e1659\") "
Oct 30 18:58:00.481172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.480399    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eaa10c34-997b-4362-92f3-e29f168e1659-kube-api-access-26dvg\" (UniqueName: \"kubernetes.io/projected/eaa10c34-997b-4362-92f3-e29f168e1659-kube-api-access-26dvg\") pod \"eaa10c34-997b-4362-92f3-e29f168e1659\" (UID: \"eaa10c34-997b-4362-92f3-e29f168e1659\") "
Oct 30 18:58:00.481172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.480443    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-metrics-certs\") pod \"db5672e5-8c89-43f8-bb92-82917a137bbd\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") "
Oct 30 18:58:00.481172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.480512    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-maas-api-tls\") pod \"db5672e5-8c89-43f8-bb92-82917a137bbd\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") "
Oct 30 18:58:00.481172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.480632    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db5672e5-8c89-43f8-bb92-82917a137bbd-kube-api-access-bvr96\" (UniqueName: \"kubernetes.io/projected/db5672e5-8c89-43f8-bb92-82917a137bbd-kube-api-access-bvr96\") pod \"db5672e5-8c89-43f8-bb92-82917a137bbd\" (UID: \"db5672e5-8c89-43f8-bb92-82917a137bbd\") "
Oct 30 18:58:00.483131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.481617    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db5672e5-8c89-43f8-bb92-82917a137bbd-openshift-service-ca" pod "db5672e5-8c89-43f8-bb92-82917a137bbd" (UID: "db5672e5-8c89-43f8-bb92-82917a137bbd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:00.483131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.482039    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaa10c34-997b-4362-92f3-e29f168e1659-plugins-config-volume" pod "eaa10c34-997b-4362-92f3-e29f168e1659" (UID: "eaa10c34-997b-4362-92f3-e29f168e1659"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:00.484226 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.484200    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-metrics-certs" pod "db5672e5-8c89-43f8-bb92-82917a137bbd" (UID: "db5672e5-8c89-43f8-bb92-82917a137bbd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:58:00.485661 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.485634    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaa10c34-997b-4362-92f3-e29f168e1659-kube-api-access-26dvg" pod "eaa10c34-997b-4362-92f3-e29f168e1659" (UID: "eaa10c34-997b-4362-92f3-e29f168e1659"). InnerVolumeSpecName "kube-api-access-26dvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:00.485877 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.485848    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db5672e5-8c89-43f8-bb92-82917a137bbd-kube-api-access-bvr96" pod "db5672e5-8c89-43f8-bb92-82917a137bbd" (UID: "db5672e5-8c89-43f8-bb92-82917a137bbd"). InnerVolumeSpecName "kube-api-access-bvr96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:00.486170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.486132    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-maas-api-tls" pod "db5672e5-8c89-43f8-bb92-82917a137bbd" (UID: "db5672e5-8c89-43f8-bb92-82917a137bbd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:58:00.497679 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.497658    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"
Oct 30 18:58:00.577876 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.577843    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:00.581773 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.581751    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0876bd1c-4d76-4118-9a9a-92c43398d4a5-kube-api-access-hnhw6\" (UniqueName: \"kubernetes.io/projected/0876bd1c-4d76-4118-9a9a-92c43398d4a5-kube-api-access-hnhw6\") pod \"0876bd1c-4d76-4118-9a9a-92c43398d4a5\" (UID: \"0876bd1c-4d76-4118-9a9a-92c43398d4a5\") "
Oct 30 18:58:00.581923 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.581800    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0876bd1c-4d76-4118-9a9a-92c43398d4a5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0876bd1c-4d76-4118-9a9a-92c43398d4a5-plugins-config-volume\") pod \"0876bd1c-4d76-4118-9a9a-92c43398d4a5\" (UID: \"0876bd1c-4d76-4118-9a9a-92c43398d4a5\") "
Oct 30 18:58:00.582268 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.582244    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0876bd1c-4d76-4118-9a9a-92c43398d4a5-plugins-config-volume" pod "0876bd1c-4d76-4118-9a9a-92c43398d4a5" (UID: "0876bd1c-4d76-4118-9a9a-92c43398d4a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:00.582479 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.582461    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26dvg\" (UniqueName: \"kubernetes.io/projected/eaa10c34-997b-4362-92f3-e29f168e1659-kube-api-access-26dvg\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:00.582538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.582483    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:00.582538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.582493    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/db5672e5-8c89-43f8-bb92-82917a137bbd-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:00.582538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.582506    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0876bd1c-4d76-4118-9a9a-92c43398d4a5-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:00.582538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.582520    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvr96\" (UniqueName: \"kubernetes.io/projected/db5672e5-8c89-43f8-bb92-82917a137bbd-kube-api-access-bvr96\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:00.582538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.582531    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/db5672e5-8c89-43f8-bb92-82917a137bbd-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:00.582538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.582539    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eaa10c34-997b-4362-92f3-e29f168e1659-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:00.584481 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.584456    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0876bd1c-4d76-4118-9a9a-92c43398d4a5-kube-api-access-hnhw6" pod "0876bd1c-4d76-4118-9a9a-92c43398d4a5" (UID: "0876bd1c-4d76-4118-9a9a-92c43398d4a5"). InnerVolumeSpecName "kube-api-access-hnhw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:00.684606 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.684574    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnhw6\" (UniqueName: \"kubernetes.io/projected/0876bd1c-4d76-4118-9a9a-92c43398d4a5-kube-api-access-hnhw6\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:00.808162 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.808077    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"
Oct 30 18:58:00.808162 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.808131    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg" event={"ID":"0876bd1c-4d76-4118-9a9a-92c43398d4a5","Type":"ContainerDied","Data":"9237c8995f00831b81a6dfb38d5b3c867e824862d1c6feb765c6b345a35ee7ab"}
Oct 30 18:58:00.808363 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.808194    4925 scope.go:122] "RemoveContainer" containerID="5dc82032fc407faaf5f7630baecfc49af458681a432fcaee4691b02ed4793f52"
Oct 30 18:58:00.809966 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.809934    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"
Oct 30 18:58:00.810173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.810038    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7" event={"ID":"db5672e5-8c89-43f8-bb92-82917a137bbd","Type":"ContainerDied","Data":"80854d619a980e6e1ac4df62046c0cba81442ca27b1b2f6ce56e8a293ae96009"}
Oct 30 18:58:00.811882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.811856    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk" event={"ID":"eaa10c34-997b-4362-92f3-e29f168e1659","Type":"ContainerDied","Data":"c4549d8ffe032c21acf177408d3cd5ad52b9b7ee0506c9039f98f879f11c5a68"}
Oct 30 18:58:00.812155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.811943    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"
Oct 30 18:58:00.822633 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.822609    4925 scope.go:122] "RemoveContainer" containerID="be4deb66846b987a9ed4b4c58951f13bb05392d97b14aad8b9dfe0f9902d9468"
Oct 30 18:58:00.835285 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.835264    4925 scope.go:122] "RemoveContainer" containerID="a607dd7582907f9cebb76ee20ccf394190022865e4c940599da5ccf5e6779acd"
Oct 30 18:58:00.841189 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.841168    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"]
Oct 30 18:58:00.845194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.845171    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e7e190-56646ff9d-plbrg"]
Oct 30 18:58:00.850537 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.850520    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"]
Oct 30 18:58:00.854960 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.854941    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e7e190-6568d5799-fnlnk"]
Oct 30 18:58:00.866790 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.866772    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"]
Oct 30 18:58:00.874746 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:00.874720    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e7e190-674dcd468f-m4lm7"]
Oct 30 18:58:01.274129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:01.272295    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:02.276534 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:02.276495    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:02.427542 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:02.427503    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0876bd1c-4d76-4118-9a9a-92c43398d4a5" path="/var/lib/kubelet/pods/0876bd1c-4d76-4118-9a9a-92c43398d4a5/volumes"
Oct 30 18:58:02.428273 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:02.428249    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db5672e5-8c89-43f8-bb92-82917a137bbd" path="/var/lib/kubelet/pods/db5672e5-8c89-43f8-bb92-82917a137bbd/volumes"
Oct 30 18:58:02.428859 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:02.428840    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eaa10c34-997b-4362-92f3-e29f168e1659" path="/var/lib/kubelet/pods/eaa10c34-997b-4362-92f3-e29f168e1659/volumes"
Oct 30 18:58:02.592209 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:02.585051    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:02.704187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:02.704158    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"
Oct 30 18:58:03.102392 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:03.102343    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:03.286754 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:03.286717    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:03.520118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:03.519742    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:03.711310 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:03.711278    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"
Oct 30 18:58:03.711727 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:03.711701    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"
Oct 30 18:58:03.712335 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:03.712311    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"
Oct 30 18:58:03.767046 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:03.766774    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 18:58:03.816328 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:03.816218    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:04.475637 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:04.472277    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:05.102165 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:05.101202    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:05.978157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:05.975682    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:06.574152 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:06.573741    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:06.810118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:06.806832    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"]
Oct 30 18:58:06.810118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:06.807129    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy" containerID="cri-o://354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300" gracePeriod=30
Oct 30 18:58:06.825166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:06.817219    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:07.014123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.013145    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:07.149156 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.149057    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"]
Oct 30 18:58:07.150132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.150085    4925 cpu_manager.go:414] "containerMap: removing container" podUID="0876bd1c-4d76-4118-9a9a-92c43398d4a5" containerName="payload-processing"
Oct 30 18:58:07.150132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.150131    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0876bd1c-4d76-4118-9a9a-92c43398d4a5" containerName="payload-processing"
Oct 30 18:58:07.150873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.150176    4925 cpu_manager.go:414] "containerMap: removing container" podUID="db5672e5-8c89-43f8-bb92-82917a137bbd" containerName="maas-api"
Oct 30 18:58:07.150873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.150185    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db5672e5-8c89-43f8-bb92-82917a137bbd" containerName="maas-api"
Oct 30 18:58:07.150873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.150226    4925 cpu_manager.go:414] "containerMap: removing container" podUID="eaa10c34-997b-4362-92f3-e29f168e1659" containerName="payload-pre-processing"
Oct 30 18:58:07.150873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.150233    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eaa10c34-997b-4362-92f3-e29f168e1659" containerName="payload-pre-processing"
Oct 30 18:58:07.150873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.150412    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="0876bd1c-4d76-4118-9a9a-92c43398d4a5" containerName="payload-processing"
Oct 30 18:58:07.150873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.150545    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="db5672e5-8c89-43f8-bb92-82917a137bbd" containerName="maas-api"
Oct 30 18:58:07.150873 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.150571    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="eaa10c34-997b-4362-92f3-e29f168e1659" containerName="payload-pre-processing"
Oct 30 18:58:07.184717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.171356    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.184717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.172622    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"]
Oct 30 18:58:07.224084 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.224057    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-fe0612-openshift-default-dockercfg-d4m8b\""
Oct 30 18:58:07.248350 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.248326    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-envoy\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.248488 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.248368    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-token\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.248488 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.248469    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-credential-socket\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.248598 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.248533    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-socket\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.248598 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.248584    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-certs\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.248691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.248624    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-data\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.248818 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.248797    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjcgv\" (UniqueName: \"kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-kube-api-access-qjcgv\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.248886 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.248842    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a67c842-7520-4af1-86b3-ef20d5bb6149-istiod-ca-cert\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.248979 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.248955    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-podinfo\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.285448 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.285419    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:07.354405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.351349    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjcgv\" (UniqueName: \"kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-kube-api-access-qjcgv\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.354405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.351421    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a67c842-7520-4af1-86b3-ef20d5bb6149-istiod-ca-cert\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.354405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.351468    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-podinfo\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.354405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.351588    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-envoy\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.354405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.351623    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-token\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.354405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.351674    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-credential-socket\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.354405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.351744    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-socket\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.354405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.351774    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-certs\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.354405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.351832    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-data\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.354405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.352360    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-data\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.354405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.353484    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a67c842-7520-4af1-86b3-ef20d5bb6149-istiod-ca-cert\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.377119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.368272    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-credential-socket\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.377119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.374523    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-socket\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.377119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.375117    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-certs\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.401180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.401041    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjcgv\" (UniqueName: \"kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-kube-api-access-qjcgv\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.409130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.408935    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-envoy\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.450126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.449315    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-podinfo\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.467122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.459571    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-token\") pod \"e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") " pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.478861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.474467    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:07.509129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.506362    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:07.882127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.881218    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:07.906510 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.906483    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"]
Oct 30 18:58:07.908720 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:58:07.908690    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a67c842_7520_4af1_86b3_ef20d5bb6149.slice/crio-3ac3e74c56f2da37eccb62a1caaac8fbe14f243bbb43e2eb9e8a774f9097c871 WatchSource:0}: Error finding container 3ac3e74c56f2da37eccb62a1caaac8fbe14f243bbb43e2eb9e8a774f9097c871: Status 404 returned error can't find the container with id 3ac3e74c56f2da37eccb62a1caaac8fbe14f243bbb43e2eb9e8a774f9097c871
Oct 30 18:58:07.911630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.911597    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:07.911727 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.911677    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:07.911727 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:07.911714    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:08.470542 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:08.470508    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:08.779206 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:08.779117    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:08.799429 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:08.799356    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"]
Oct 30 18:58:08.805156 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:08.805125    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:08.855898 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:08.855865    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-655193-openshift-default-dockercfg-8sbqd\""
Oct 30 18:58:08.859166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:08.859137    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"]
Oct 30 18:58:09.050170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.049414    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:09.076124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.074080    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/415056de-18a9-46fc-97cd-db55c15ce521-istio-podinfo\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.076124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.075868    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-certs\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.076124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.075915    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-data\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.076124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.075947    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/415056de-18a9-46fc-97cd-db55c15ce521-istiod-ca-cert\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.076124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.075981    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-credential-socket\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.076124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.076034    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-istio-token\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.076561 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.076237    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkl8b\" (UniqueName: \"kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-kube-api-access-rkl8b\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.076561 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.076287    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-envoy\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.076561 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.076400    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-socket\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.117119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.086808    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" event={"ID":"0a67c842-7520-4af1-86b3-ef20d5bb6149","Type":"ContainerStarted","Data":"59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d"}
Oct 30 18:58:09.117119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.086878    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" event={"ID":"0a67c842-7520-4af1-86b3-ef20d5bb6149","Type":"ContainerStarted","Data":"3ac3e74c56f2da37eccb62a1caaac8fbe14f243bbb43e2eb9e8a774f9097c871"}
Oct 30 18:58:09.209129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.208664    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkl8b\" (UniqueName: \"kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-kube-api-access-rkl8b\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.209129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.208722    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-envoy\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.209129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.208821    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-socket\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.209129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.208932    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/415056de-18a9-46fc-97cd-db55c15ce521-istio-podinfo\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.209129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.209037    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-certs\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.209129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.209064    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-data\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.209129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.209111    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/415056de-18a9-46fc-97cd-db55c15ce521-istiod-ca-cert\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.209129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.209145    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-credential-socket\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.213537 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.209186    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-istio-token\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.219942 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.213976    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-data\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.219942 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.218715    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/415056de-18a9-46fc-97cd-db55c15ce521-istiod-ca-cert\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.219942 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.218973    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-credential-socket\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.219942 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.219909    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-certs\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.221617 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.221064    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-socket\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.390422 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.390290    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:09.444216 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.444172    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/415056de-18a9-46fc-97cd-db55c15ce521-istio-podinfo\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.445751 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.445717    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkl8b\" (UniqueName: \"kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-kube-api-access-rkl8b\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.448444 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.448343    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-envoy\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.453317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.453286    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-istio-token\") pod \"e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") " pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.506907 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.506866    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:09.512768 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.512745    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:09.532934 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.532891    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" podStartSLOduration=2.532878442 podStartE2EDuration="2.532878442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:09.149903208 +0000 UTC m=+1310.866248286" watchObservedRunningTime="2026-10-30 18:58:09.532878442 +0000 UTC m=+1311.249223517"
Oct 30 18:58:09.581026 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.580993    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:09.721422 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.721393    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:09.753005 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.752974    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 18:58:09.753170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:09.753041    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:58:10.036170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:10.035739    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"]
Oct 30 18:58:10.073139 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:10.072826    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:10.101943 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:58:10.100931    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod415056de_18a9_46fc_97cd_db55c15ce521.slice/crio-d174970ea02168ec92d18d818a4eaf1b7c55b7251f87a1f025961871b0f1d0d1 WatchSource:0}: Error finding container d174970ea02168ec92d18d818a4eaf1b7c55b7251f87a1f025961871b0f1d0d1: Status 404 returned error can't find the container with id d174970ea02168ec92d18d818a4eaf1b7c55b7251f87a1f025961871b0f1d0d1
Oct 30 18:58:10.108601 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:10.104372    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:10.108601 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:10.104464    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:10.108601 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:10.104509    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:10.108601 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:10.107894    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:10.114428 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:10.114405    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:58:10.262538 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:10.262470    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:10.558165 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:10.558132    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:10.926741 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:10.926708    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:11.115119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:11.111744    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:11.200173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:11.191635    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" event={"ID":"415056de-18a9-46fc-97cd-db55c15ce521","Type":"ContainerStarted","Data":"9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7"}
Oct 30 18:58:11.200173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:11.191689    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" event={"ID":"415056de-18a9-46fc-97cd-db55c15ce521","Type":"ContainerStarted","Data":"d174970ea02168ec92d18d818a4eaf1b7c55b7251f87a1f025961871b0f1d0d1"}
Oct 30 18:58:11.236783 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:11.231882    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" podStartSLOduration=3.23186126 podStartE2EDuration="3.23186126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:11.219769561 +0000 UTC m=+1312.936114638" watchObservedRunningTime="2026-10-30 18:58:11.23186126 +0000 UTC m=+1312.948206337"
Oct 30 18:58:11.315342 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:11.315306    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:11.520174 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:11.513639    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:11.768118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:11.732240    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:11.768118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:11.741716    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:11.843242 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:11.843161    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:11.947766 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:11.947593    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:12.142620 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.142594    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:58:12.146574 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.146477    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:12.188495 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.188463    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-credential-socket\") pod \"17503242-eada-4ef5-b87a-8c766b072494\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") "
Oct 30 18:58:12.188689 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.188560    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17503242-eada-4ef5-b87a-8c766b072494-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17503242-eada-4ef5-b87a-8c766b072494-istiod-ca-cert\") pod \"17503242-eada-4ef5-b87a-8c766b072494\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") "
Oct 30 18:58:12.188689 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.188622    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-kube-api-access-7zc95\" (UniqueName: \"kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-kube-api-access-7zc95\") pod \"17503242-eada-4ef5-b87a-8c766b072494\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") "
Oct 30 18:58:12.188689 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.188663    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-socket\") pod \"17503242-eada-4ef5-b87a-8c766b072494\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") "
Oct 30 18:58:12.188826 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.188768    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-certs\") pod \"17503242-eada-4ef5-b87a-8c766b072494\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") "
Oct 30 18:58:12.188870 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.188824    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-data\") pod \"17503242-eada-4ef5-b87a-8c766b072494\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") "
Oct 30 18:58:12.188911 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.188872    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-istio-token\" (UniqueName: \"kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-istio-token\") pod \"17503242-eada-4ef5-b87a-8c766b072494\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") "
Oct 30 18:58:12.188986 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.188960    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/17503242-eada-4ef5-b87a-8c766b072494-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17503242-eada-4ef5-b87a-8c766b072494-istio-podinfo\") pod \"17503242-eada-4ef5-b87a-8c766b072494\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") "
Oct 30 18:58:12.189048 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.189010    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-envoy\") pod \"17503242-eada-4ef5-b87a-8c766b072494\" (UID: \"17503242-eada-4ef5-b87a-8c766b072494\") "
Oct 30 18:58:12.189710 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.189489    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17503242-eada-4ef5-b87a-8c766b072494-istiod-ca-cert" pod "17503242-eada-4ef5-b87a-8c766b072494" (UID: "17503242-eada-4ef5-b87a-8c766b072494"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:12.189863 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.189788    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-credential-socket" pod "17503242-eada-4ef5-b87a-8c766b072494" (UID: "17503242-eada-4ef5-b87a-8c766b072494"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:12.189863 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.189815    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17503242-eada-4ef5-b87a-8c766b072494-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:12.190077 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.190042    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-data" pod "17503242-eada-4ef5-b87a-8c766b072494" (UID: "17503242-eada-4ef5-b87a-8c766b072494"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:12.190521 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.190501    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-socket" pod "17503242-eada-4ef5-b87a-8c766b072494" (UID: "17503242-eada-4ef5-b87a-8c766b072494"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:12.192408 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.192376    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-certs" pod "17503242-eada-4ef5-b87a-8c766b072494" (UID: "17503242-eada-4ef5-b87a-8c766b072494"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:12.194767 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.194738    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-kube-api-access-7zc95" pod "17503242-eada-4ef5-b87a-8c766b072494" (UID: "17503242-eada-4ef5-b87a-8c766b072494"). InnerVolumeSpecName "kube-api-access-7zc95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:12.194880 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.194738    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17503242-eada-4ef5-b87a-8c766b072494-istio-podinfo" pod "17503242-eada-4ef5-b87a-8c766b072494" (UID: "17503242-eada-4ef5-b87a-8c766b072494"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:58:12.195364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.195330    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-istio-token" pod "17503242-eada-4ef5-b87a-8c766b072494" (UID: "17503242-eada-4ef5-b87a-8c766b072494"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:12.195551 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.195530    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-envoy" pod "17503242-eada-4ef5-b87a-8c766b072494" (UID: "17503242-eada-4ef5-b87a-8c766b072494"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:12.203519 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.203492    4925 generic.go:356] "Generic (PLEG): container finished" podID="17503242-eada-4ef5-b87a-8c766b072494" containerID="354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300" exitCode=0
Oct 30 18:58:12.203648 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.203626    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"
Oct 30 18:58:12.203720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.203638    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" event={"ID":"17503242-eada-4ef5-b87a-8c766b072494","Type":"ContainerDied","Data":"354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300"}
Oct 30 18:58:12.203720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.203706    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg" event={"ID":"17503242-eada-4ef5-b87a-8c766b072494","Type":"ContainerDied","Data":"23b4423a77c9ac42d533fedb8dcb890d60bb2f29eb7cbeb67867816c8ebc5602"}
Oct 30 18:58:12.203827 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.203738    4925 scope.go:122] "RemoveContainer" containerID="354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300"
Oct 30 18:58:12.204559 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.204436    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:12.206330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.205464    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:12.303514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.300690    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:12.303514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.300724    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zc95\" (UniqueName: \"kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-kube-api-access-7zc95\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:12.303514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.300740    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:12.303514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.300754    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:12.303514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.300769    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:12.303514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.300782    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17503242-eada-4ef5-b87a-8c766b072494-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:12.303514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.300795    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17503242-eada-4ef5-b87a-8c766b072494-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:12.303514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.300808    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17503242-eada-4ef5-b87a-8c766b072494-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:12.325289 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.325258    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"]
Oct 30 18:58:12.361676 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.361595    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-e7e190-openshift-default-58b77988c5-qfwrg"]
Oct 30 18:58:12.399383 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.373665    4925 scope.go:122] "RemoveContainer" containerID="da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192"
Oct 30 18:58:12.567454 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.567420    4925 scope.go:122] "RemoveContainer" containerID="354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300"
Oct 30 18:58:12.569701 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:58:12.569655    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300\": container with ID starting with 354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300 not found: ID does not exist" containerID="354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300"
Oct 30 18:58:12.570742 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.570707    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300"} err="failed to get container status \"354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300\": rpc error: code = NotFound desc = could not find container \"354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300\": container with ID starting with 354e63a63d7bcec3c1c86e1c3358943d43a17dcafb3d160d6c040bafae197300 not found: ID does not exist"
Oct 30 18:58:12.570894 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.570882    4925 scope.go:122] "RemoveContainer" containerID="da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192"
Oct 30 18:58:12.571832 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:58:12.571756    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192\": container with ID starting with da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192 not found: ID does not exist" containerID="da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192"
Oct 30 18:58:12.571832 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.571793    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192"} err="failed to get container status \"da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192\": rpc error: code = NotFound desc = could not find container \"da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192\": container with ID starting with da59bef2401d24b600160506f6a5aca11ca5623f49c218da81f683e1a0238192 not found: ID does not exist"
Oct 30 18:58:12.573000 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.572975    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17503242-eada-4ef5-b87a-8c766b072494" path="/var/lib/kubelet/pods/17503242-eada-4ef5-b87a-8c766b072494/volumes"
Oct 30 18:58:12.838552 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:12.836194    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:13.121807 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.121734    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"]
Oct 30 18:58:13.122436 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.122418    4925 cpu_manager.go:414] "containerMap: removing container" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy"
Oct 30 18:58:13.122436 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.122434    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy"
Oct 30 18:58:13.122571 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.122446    4925 cpu_manager.go:414] "containerMap: removing container" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy"
Oct 30 18:58:13.122571 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.122451    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy"
Oct 30 18:58:13.122571 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.122563    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy"
Oct 30 18:58:13.122571 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.122571    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="17503242-eada-4ef5-b87a-8c766b072494" containerName="istio-proxy"
Oct 30 18:58:13.127953 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.127935    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"
Oct 30 18:58:13.134699 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.134666    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"]
Oct 30 18:58:13.136771 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.136608    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-fe0612\""
Oct 30 18:58:13.148125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.145206    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"]
Oct 30 18:58:13.159137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.150361    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"
Oct 30 18:58:13.159137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.156273    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-fe0612-dockercfg-nhxtw\""
Oct 30 18:58:13.183128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.182141    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:13.183128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.182302    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"]
Oct 30 18:58:13.278131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.269657    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2nnc\" (UniqueName: \"kubernetes.io/projected/73b71a46-e413-45f2-a484-415344e068d3-kube-api-access-r2nnc\") pod \"payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg\" (UID: \"73b71a46-e413-45f2-a484-415344e068d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"
Oct 30 18:58:13.278131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.269743    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjmb2\" (UniqueName: \"kubernetes.io/projected/6ac6b3d6-6972-4721-af33-90a13e518e4e-kube-api-access-fjmb2\") pod \"payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf\" (UID: \"6ac6b3d6-6972-4721-af33-90a13e518e4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"
Oct 30 18:58:13.278131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.269861    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b71a46-e413-45f2-a484-415344e068d3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg\" (UID: \"73b71a46-e413-45f2-a484-415344e068d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"
Oct 30 18:58:13.278131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.270001    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ac6b3d6-6972-4721-af33-90a13e518e4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf\" (UID: \"6ac6b3d6-6972-4721-af33-90a13e518e4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"
Oct 30 18:58:13.365582 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.361119    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:13.392195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.381263    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2nnc\" (UniqueName: \"kubernetes.io/projected/73b71a46-e413-45f2-a484-415344e068d3-kube-api-access-r2nnc\") pod \"payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg\" (UID: \"73b71a46-e413-45f2-a484-415344e068d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"
Oct 30 18:58:13.392195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.381337    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjmb2\" (UniqueName: \"kubernetes.io/projected/6ac6b3d6-6972-4721-af33-90a13e518e4e-kube-api-access-fjmb2\") pod \"payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf\" (UID: \"6ac6b3d6-6972-4721-af33-90a13e518e4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"
Oct 30 18:58:13.392195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.381435    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b71a46-e413-45f2-a484-415344e068d3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg\" (UID: \"73b71a46-e413-45f2-a484-415344e068d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"
Oct 30 18:58:13.392195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.381538    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ac6b3d6-6972-4721-af33-90a13e518e4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf\" (UID: \"6ac6b3d6-6972-4721-af33-90a13e518e4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"
Oct 30 18:58:13.392195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.390822    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b71a46-e413-45f2-a484-415344e068d3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg\" (UID: \"73b71a46-e413-45f2-a484-415344e068d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"
Oct 30 18:58:13.392195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.391222    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ac6b3d6-6972-4721-af33-90a13e518e4e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf\" (UID: \"6ac6b3d6-6972-4721-af33-90a13e518e4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"
Oct 30 18:58:13.428062 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.422475    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjmb2\" (UniqueName: \"kubernetes.io/projected/6ac6b3d6-6972-4721-af33-90a13e518e4e-kube-api-access-fjmb2\") pod \"payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf\" (UID: \"6ac6b3d6-6972-4721-af33-90a13e518e4e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"
Oct 30 18:58:13.465118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.442783    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2nnc\" (UniqueName: \"kubernetes.io/projected/73b71a46-e413-45f2-a484-415344e068d3-kube-api-access-r2nnc\") pod \"payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg\" (UID: \"73b71a46-e413-45f2-a484-415344e068d3\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"
Oct 30 18:58:13.496120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.493323    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"
Oct 30 18:58:13.499421 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.499328    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"
Oct 30 18:58:13.556307 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.556206    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:13.643172 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:58:13.641656    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:58:13.818260 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.818081    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:13.819466 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.819437    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"]
Oct 30 18:58:13.830752 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.830718    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"]
Oct 30 18:58:13.831052 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.830995    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:13.835086 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.835069    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-f778f5-openshift-default-dockercfg-wpmgs\""
Oct 30 18:58:13.913128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.912034    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d699d780-921f-48ae-8ce4-27fdd806b52d-istiod-ca-cert\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:13.913128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.912113    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-podinfo\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:13.913128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.912159    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzpb4\" (UniqueName: \"kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-kube-api-access-lzpb4\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:13.913128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.912204    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-envoy\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:13.913128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.912259    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-socket\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:13.913128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.912283    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-data\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:13.913128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.912334    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-credential-socket\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:13.913128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.912396    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-certs\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:13.913128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:13.912541    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-token\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.054351 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.041992    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-data\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.054670 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.053891    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-data\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.055060 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.054785    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-credential-socket\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.055311 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.055292    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-certs\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.064338 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.055736    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-certs\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.064600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.060406    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:14.066409 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.055004    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-credential-socket\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.080135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.079324    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-token\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.093826 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.093793    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d699d780-921f-48ae-8ce4-27fdd806b52d-istiod-ca-cert\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.094016 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.093866    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-podinfo\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.094016 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.093936    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzpb4\" (UniqueName: \"kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-kube-api-access-lzpb4\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.094016 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.094002    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-envoy\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.094214 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.094081    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-socket\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.095289 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.094608    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-socket\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.095289 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.095230    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d699d780-921f-48ae-8ce4-27fdd806b52d-istiod-ca-cert\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.096879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.096836    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-token\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.118587 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.118554    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-envoy\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.119619 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.119314    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-podinfo\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.125456 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.125390    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzpb4\" (UniqueName: \"kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-kube-api-access-lzpb4\") pod \"e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") " pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.158815 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.158734    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:14.331698 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.330114    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:14.528077 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.528042    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:14.566142 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.553379    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"]
Oct 30 18:58:14.601743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.601650    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"]
Oct 30 18:58:14.940466 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.940427    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"]
Oct 30 18:58:14.944562 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:58:14.944529    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd699d780_921f_48ae_8ce4_27fdd806b52d.slice/crio-776ff6f03a1675686f4cd1bb9220b04873740fb1ed639e2374da0642c888312d WatchSource:0}: Error finding container 776ff6f03a1675686f4cd1bb9220b04873740fb1ed639e2374da0642c888312d: Status 404 returned error can't find the container with id 776ff6f03a1675686f4cd1bb9220b04873740fb1ed639e2374da0642c888312d
Oct 30 18:58:14.951250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.950399    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:14.951250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.950495    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:14.951250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:14.950535    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:15.111699 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.103663    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"]
Oct 30 18:58:15.126129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.113988    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"
Oct 30 18:58:15.126129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.116934    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:15.126129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.117782    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"]
Oct 30 18:58:15.147601 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.147567    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"]
Oct 30 18:58:15.505173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.490026    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdz4c\" (UniqueName: \"kubernetes.io/projected/ad8ce530-37d0-48f6-805d-1b034492b4fe-kube-api-access-zdz4c\") pod \"payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj\" (UID: \"ad8ce530-37d0-48f6-805d-1b034492b4fe\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"
Oct 30 18:58:15.505173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.490308    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8ce530-37d0-48f6-805d-1b034492b4fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj\" (UID: \"ad8ce530-37d0-48f6-805d-1b034492b4fe\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"
Oct 30 18:58:15.505173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.493757    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-655193-dockercfg-jmqr2\""
Oct 30 18:58:15.546869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.532414    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-655193\""
Oct 30 18:58:15.554240 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.547909    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:15.599791 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.599760    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"
Oct 30 18:58:15.604531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.604498    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdz4c\" (UniqueName: \"kubernetes.io/projected/ad8ce530-37d0-48f6-805d-1b034492b4fe-kube-api-access-zdz4c\") pod \"payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj\" (UID: \"ad8ce530-37d0-48f6-805d-1b034492b4fe\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"
Oct 30 18:58:15.604911 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.604886    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8ce530-37d0-48f6-805d-1b034492b4fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj\" (UID: \"ad8ce530-37d0-48f6-805d-1b034492b4fe\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"
Oct 30 18:58:15.609232 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.608824    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8ce530-37d0-48f6-805d-1b034492b4fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj\" (UID: \"ad8ce530-37d0-48f6-805d-1b034492b4fe\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"
Oct 30 18:58:15.610323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.609535    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"]
Oct 30 18:58:15.631578 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.630858    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf" event={"ID":"6ac6b3d6-6972-4721-af33-90a13e518e4e","Type":"ContainerStarted","Data":"98cec756b152b16181640fe95b3aad1e7d0d276971579e50b58d7408dfd0c452"}
Oct 30 18:58:15.631578 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.631031    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"
Oct 30 18:58:15.631578 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.631163    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf" event={"ID":"6ac6b3d6-6972-4721-af33-90a13e518e4e","Type":"ContainerStarted","Data":"de30f07373ccc4b1261e972fb1da3d2a892144840d28a8cd9cdebeb79ba40b0f"}
Oct 30 18:58:15.634289 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.634248    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" event={"ID":"d699d780-921f-48ae-8ce4-27fdd806b52d","Type":"ContainerStarted","Data":"07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858"}
Oct 30 18:58:15.634519 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.634499    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" event={"ID":"d699d780-921f-48ae-8ce4-27fdd806b52d","Type":"ContainerStarted","Data":"776ff6f03a1675686f4cd1bb9220b04873740fb1ed639e2374da0642c888312d"}
Oct 30 18:58:15.640132 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.638599    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdz4c\" (UniqueName: \"kubernetes.io/projected/ad8ce530-37d0-48f6-805d-1b034492b4fe-kube-api-access-zdz4c\") pod \"payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj\" (UID: \"ad8ce530-37d0-48f6-805d-1b034492b4fe\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"
Oct 30 18:58:15.643168 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.643121    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg" event={"ID":"73b71a46-e413-45f2-a484-415344e068d3","Type":"ContainerStarted","Data":"efb7619b2cb6869a2e0402fe1c5892749f3f58e51a0c9f1a15625726b72ef065"}
Oct 30 18:58:15.643341 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.643319    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg" event={"ID":"73b71a46-e413-45f2-a484-415344e068d3","Type":"ContainerStarted","Data":"61dcb3aa2b57e7d51cf6f2720a0241b3d881af0408e6f6c9d6e2dd11107ecd0d"}
Oct 30 18:58:15.643420 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.643369    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"
Oct 30 18:58:15.681267 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.656408    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf" podStartSLOduration=2.656393262 podStartE2EDuration="2.656393262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:15.652744771 +0000 UTC m=+1317.369089847" watchObservedRunningTime="2026-10-30 18:58:15.656393262 +0000 UTC m=+1317.372738339"
Oct 30 18:58:15.714143 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.713163    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcvpc\" (UniqueName: \"kubernetes.io/projected/fd35ca47-1812-4719-ad92-5993b466e8c2-kube-api-access-xcvpc\") pod \"payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh\" (UID: \"fd35ca47-1812-4719-ad92-5993b466e8c2\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"
Oct 30 18:58:15.714143 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.713683    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd35ca47-1812-4719-ad92-5993b466e8c2-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh\" (UID: \"fd35ca47-1812-4719-ad92-5993b466e8c2\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"
Oct 30 18:58:15.726428 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.726367    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg" podStartSLOduration=2.726350119 podStartE2EDuration="2.726350119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:15.701753157 +0000 UTC m=+1317.418098226" watchObservedRunningTime="2026-10-30 18:58:15.726350119 +0000 UTC m=+1317.442695197"
Oct 30 18:58:15.726720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.726682    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" podStartSLOduration=2.726673433 podStartE2EDuration="2.726673433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:15.721444648 +0000 UTC m=+1317.437789724" watchObservedRunningTime="2026-10-30 18:58:15.726673433 +0000 UTC m=+1317.443018510"
Oct 30 18:58:15.815934 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.815854    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd35ca47-1812-4719-ad92-5993b466e8c2-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh\" (UID: \"fd35ca47-1812-4719-ad92-5993b466e8c2\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"
Oct 30 18:58:15.816086 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.815987    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcvpc\" (UniqueName: \"kubernetes.io/projected/fd35ca47-1812-4719-ad92-5993b466e8c2-kube-api-access-xcvpc\") pod \"payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh\" (UID: \"fd35ca47-1812-4719-ad92-5993b466e8c2\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"
Oct 30 18:58:15.817112 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.817063    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd35ca47-1812-4719-ad92-5993b466e8c2-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh\" (UID: \"fd35ca47-1812-4719-ad92-5993b466e8c2\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"
Oct 30 18:58:15.825040 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.825016    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcvpc\" (UniqueName: \"kubernetes.io/projected/fd35ca47-1812-4719-ad92-5993b466e8c2-kube-api-access-xcvpc\") pod \"payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh\" (UID: \"fd35ca47-1812-4719-ad92-5993b466e8c2\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"
Oct 30 18:58:15.848326 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.848289    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"
Oct 30 18:58:15.958423 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:15.958381    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"
Oct 30 18:58:16.170183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:16.170152    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:16.210421 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:16.210381    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:16.538161 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:16.526281    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"]
Oct 30 18:58:16.542120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:16.541196    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"]
Oct 30 18:58:16.553178 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:58:16.550480    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad8ce530_37d0_48f6_805d_1b034492b4fe.slice/crio-c0c35ba8d2805a06108ba9270efae65f2a0ec474780c3d3c3b596c3565b4cfee WatchSource:0}: Error finding container c0c35ba8d2805a06108ba9270efae65f2a0ec474780c3d3c3b596c3565b4cfee: Status 404 returned error can't find the container with id c0c35ba8d2805a06108ba9270efae65f2a0ec474780c3d3c3b596c3565b4cfee
Oct 30 18:58:16.654184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:16.654159    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj" event={"ID":"ad8ce530-37d0-48f6-805d-1b034492b4fe","Type":"ContainerStarted","Data":"c0c35ba8d2805a06108ba9270efae65f2a0ec474780c3d3c3b596c3565b4cfee"}
Oct 30 18:58:16.655650 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:16.655622    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh" event={"ID":"fd35ca47-1812-4719-ad92-5993b466e8c2","Type":"ContainerStarted","Data":"5affec9414e02447731ad2d48207c9878142045fa8bbdd369bae2536b4ea9a32"}
Oct 30 18:58:16.679115 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:16.656908    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:16.679364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:16.679347    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:58:16.912814 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:16.912713    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:17.015640 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.015599    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"]
Oct 30 18:58:17.020962 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.020935    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"
Oct 30 18:58:17.027263 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.027227    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-a014d1\""
Oct 30 18:58:17.028195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.028035    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-a014d1-dockercfg-wqcqs\""
Oct 30 18:58:17.033897 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.033872    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"]
Oct 30 18:58:17.055415 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.055391    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"]
Oct 30 18:58:17.095161 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.093354    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"
Oct 30 18:58:17.107121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.105358    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"]
Oct 30 18:58:17.176123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.159430    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:17.318683 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.304320    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc42e32a-caae-45ee-a16c-bf281e087e1b-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9\" (UID: \"fc42e32a-caae-45ee-a16c-bf281e087e1b\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"
Oct 30 18:58:17.318683 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.304423    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbnwv\" (UniqueName: \"kubernetes.io/projected/fc42e32a-caae-45ee-a16c-bf281e087e1b-kube-api-access-mbnwv\") pod \"payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9\" (UID: \"fc42e32a-caae-45ee-a16c-bf281e087e1b\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"
Oct 30 18:58:17.412820 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.405806    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-295t7\" (UniqueName: \"kubernetes.io/projected/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-kube-api-access-295t7\") pod \"payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk\" (UID: \"dd91f256-8f26-4ed2-97c7-c2835bb0f90a\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"
Oct 30 18:58:17.412820 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.405983    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk\" (UID: \"dd91f256-8f26-4ed2-97c7-c2835bb0f90a\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"
Oct 30 18:58:17.412820 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.406054    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc42e32a-caae-45ee-a16c-bf281e087e1b-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9\" (UID: \"fc42e32a-caae-45ee-a16c-bf281e087e1b\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"
Oct 30 18:58:17.412820 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.406139    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbnwv\" (UniqueName: \"kubernetes.io/projected/fc42e32a-caae-45ee-a16c-bf281e087e1b-kube-api-access-mbnwv\") pod \"payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9\" (UID: \"fc42e32a-caae-45ee-a16c-bf281e087e1b\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"
Oct 30 18:58:17.412820 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.407535    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc42e32a-caae-45ee-a16c-bf281e087e1b-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9\" (UID: \"fc42e32a-caae-45ee-a16c-bf281e087e1b\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"
Oct 30 18:58:17.441155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.435828    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbnwv\" (UniqueName: \"kubernetes.io/projected/fc42e32a-caae-45ee-a16c-bf281e087e1b-kube-api-access-mbnwv\") pod \"payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9\" (UID: \"fc42e32a-caae-45ee-a16c-bf281e087e1b\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"
Oct 30 18:58:17.516008 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.515956    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-295t7\" (UniqueName: \"kubernetes.io/projected/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-kube-api-access-295t7\") pod \"payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk\" (UID: \"dd91f256-8f26-4ed2-97c7-c2835bb0f90a\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"
Oct 30 18:58:17.516370 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.516349    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk\" (UID: \"dd91f256-8f26-4ed2-97c7-c2835bb0f90a\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"
Oct 30 18:58:17.518655 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.518620    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk\" (UID: \"dd91f256-8f26-4ed2-97c7-c2835bb0f90a\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"
Oct 30 18:58:17.531042 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.531010    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-295t7\" (UniqueName: \"kubernetes.io/projected/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-kube-api-access-295t7\") pod \"payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk\" (UID: \"dd91f256-8f26-4ed2-97c7-c2835bb0f90a\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"
Oct 30 18:58:17.639311 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.639282    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"
Oct 30 18:58:17.701139 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.700853    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"
Oct 30 18:58:17.718166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.712896    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj" event={"ID":"ad8ce530-37d0-48f6-805d-1b034492b4fe","Type":"ContainerStarted","Data":"d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192"}
Oct 30 18:58:17.718166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.713606    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"
Oct 30 18:58:17.724831 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.723953    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh" event={"ID":"fd35ca47-1812-4719-ad92-5993b466e8c2","Type":"ContainerStarted","Data":"ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c"}
Oct 30 18:58:17.740413 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.740147    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj" podStartSLOduration=2.740072142 podStartE2EDuration="2.740072142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:17.735990523 +0000 UTC m=+1319.452335599" watchObservedRunningTime="2026-10-30 18:58:17.740072142 +0000 UTC m=+1319.456417219"
Oct 30 18:58:17.777260 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:17.755561    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh" podStartSLOduration=2.755494958 podStartE2EDuration="2.755494958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:17.751749203 +0000 UTC m=+1319.468094270" watchObservedRunningTime="2026-10-30 18:58:17.755494958 +0000 UTC m=+1319.471840035"
Oct 30 18:58:18.010170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.009773    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:18.123882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.123724    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"]
Oct 30 18:58:18.129266 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.129244    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"]
Oct 30 18:58:18.147321 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.147189    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"]
Oct 30 18:58:18.214050 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.213944    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"]
Oct 30 18:58:18.225224 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.225193    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.231944 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.231083    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-93574a-openshift-default-dockercfg-d46c7\""
Oct 30 18:58:18.336336 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.336265    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:18.390753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.390121    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-envoy\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.390753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.390217    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69bfg\" (UniqueName: \"kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-kube-api-access-69bfg\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.390753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.390296    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-certs\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.390753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.390329    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istiod-ca-cert\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.390753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.390390    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-credential-socket\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.390753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.390413    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-token\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.390753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.390443    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-socket\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.390753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.390504    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-data\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.390753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.390535    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-podinfo\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.495225 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.495184    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-envoy\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.495392 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.495276    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69bfg\" (UniqueName: \"kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-kube-api-access-69bfg\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.495392 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.495366    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-certs\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.495480 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.495399    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istiod-ca-cert\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.495480 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.495464    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-credential-socket\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.495558 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.495488    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-token\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.495558 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.495520    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-socket\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.496063 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.496037    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-data\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.496114 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.496087    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-podinfo\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.497732 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.497705    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istiod-ca-cert\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.499776 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.498573    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-credential-socket\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.499776 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.499088    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-certs\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.499776 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.499404    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-data\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.499776 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.499406    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-socket\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.500621 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.500581    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-envoy\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.500753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.500730    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-podinfo\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.521337 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.521282    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-token\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.521693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.521672    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69bfg\" (UniqueName: \"kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-kube-api-access-69bfg\") pod \"e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") " pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.527626 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.527604    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:18.617859 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.617835    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-93574a-openshift-default-dockercfg-d46c7\""
Oct 30 18:58:18.623586 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.623553    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:18.737195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.736978    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk" event={"ID":"dd91f256-8f26-4ed2-97c7-c2835bb0f90a","Type":"ContainerStarted","Data":"a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e"}
Oct 30 18:58:18.737195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.737049    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk" event={"ID":"dd91f256-8f26-4ed2-97c7-c2835bb0f90a","Type":"ContainerStarted","Data":"182d464a94310fa6cab3ce893f6c24ac69511673742ee367f5d1019dac956254"}
Oct 30 18:58:18.737195 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.737154    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"
Oct 30 18:58:18.739340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.739258    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9" event={"ID":"fc42e32a-caae-45ee-a16c-bf281e087e1b","Type":"ContainerStarted","Data":"b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367"}
Oct 30 18:58:18.739340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.739306    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9" event={"ID":"fc42e32a-caae-45ee-a16c-bf281e087e1b","Type":"ContainerStarted","Data":"5f8a3636d496c5bc3e7017623b5400aafd201bc053efab12ce853b47c7b985a9"}
Oct 30 18:58:18.739876 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.739557    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"
Oct 30 18:58:18.740524 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.740497    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"
Oct 30 18:58:18.810970 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.798991    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk" podStartSLOduration=2.798971313 podStartE2EDuration="2.798971313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:18.753906381 +0000 UTC m=+1320.470251457" watchObservedRunningTime="2026-10-30 18:58:18.798971313 +0000 UTC m=+1320.515316391"
Oct 30 18:58:18.810970 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:18.810717    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9" podStartSLOduration=1.81070187 podStartE2EDuration="1.81070187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:18.810616193 +0000 UTC m=+1320.526961295" watchObservedRunningTime="2026-10-30 18:58:18.81070187 +0000 UTC m=+1320.527046947"
Oct 30 18:58:19.049311 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.046158    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"]
Oct 30 18:58:19.050141 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:58:19.050088    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb5bb8c_bb43_4b3b_84de_1f1793672d4b.slice/crio-7f83518d9f890a90f97ba1bc0fdd9c687c177eb92cae863df5dd8568167e0510 WatchSource:0}: Error finding container 7f83518d9f890a90f97ba1bc0fdd9c687c177eb92cae863df5dd8568167e0510: Status 404 returned error can't find the container with id 7f83518d9f890a90f97ba1bc0fdd9c687c177eb92cae863df5dd8568167e0510
Oct 30 18:58:19.052972 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.052939    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:19.053029 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.053009    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:19.053074 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.053037    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:19.202074 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.202039    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:19.636489 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.636455    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:19.682435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.682388    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"]
Oct 30 18:58:19.705531 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.705493    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"
Oct 30 18:58:19.812313 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.803264    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-f778f5-dockercfg-b2dw7\""
Oct 30 18:58:19.842033 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.814569    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-f778f5\""
Oct 30 18:58:19.842297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.815560    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjnwg\" (UniqueName: \"kubernetes.io/projected/4cb3fee4-1878-4b77-8ee6-ec9767749621-kube-api-access-pjnwg\") pod \"payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t\" (UID: \"4cb3fee4-1878-4b77-8ee6-ec9767749621\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"
Oct 30 18:58:19.842866 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.842842    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb3fee4-1878-4b77-8ee6-ec9767749621-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t\" (UID: \"4cb3fee4-1878-4b77-8ee6-ec9767749621\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"
Oct 30 18:58:19.980573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.977227    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:19.980963 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.980932    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"]
Oct 30 18:58:19.983514 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.983484    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb3fee4-1878-4b77-8ee6-ec9767749621-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t\" (UID: \"4cb3fee4-1878-4b77-8ee6-ec9767749621\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"
Oct 30 18:58:19.987240 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.983711    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjnwg\" (UniqueName: \"kubernetes.io/projected/4cb3fee4-1878-4b77-8ee6-ec9767749621-kube-api-access-pjnwg\") pod \"payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t\" (UID: \"4cb3fee4-1878-4b77-8ee6-ec9767749621\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"
Oct 30 18:58:19.987240 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:19.985515    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb3fee4-1878-4b77-8ee6-ec9767749621-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t\" (UID: \"4cb3fee4-1878-4b77-8ee6-ec9767749621\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"
Oct 30 18:58:20.118530 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:20.118491    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjnwg\" (UniqueName: \"kubernetes.io/projected/4cb3fee4-1878-4b77-8ee6-ec9767749621-kube-api-access-pjnwg\") pod \"payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t\" (UID: \"4cb3fee4-1878-4b77-8ee6-ec9767749621\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"
Oct 30 18:58:20.121391 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:20.121353    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82" event={"ID":"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b","Type":"ContainerStarted","Data":"ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f"}
Oct 30 18:58:20.121550 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:20.121452    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82" event={"ID":"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b","Type":"ContainerStarted","Data":"7f83518d9f890a90f97ba1bc0fdd9c687c177eb92cae863df5dd8568167e0510"}
Oct 30 18:58:20.162372 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:20.149049    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82" podStartSLOduration=2.149032089 podStartE2EDuration="2.149032089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:20.142941016 +0000 UTC m=+1321.859286097" watchObservedRunningTime="2026-10-30 18:58:20.149032089 +0000 UTC m=+1321.865377168"
Oct 30 18:58:20.283843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:20.283752    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"
Oct 30 18:58:20.386523 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:20.386433    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:20.624249 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:20.624160    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:20.625530 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:20.625497    4925 patch_prober.go:29] interesting pod/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Oct 30 18:58:20.625655 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:20.625562    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82" podUID="9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Oct 30 18:58:20.695789 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:20.695289    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"]
Oct 30 18:58:21.049825 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:21.049783    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t" event={"ID":"4cb3fee4-1878-4b77-8ee6-ec9767749621","Type":"ContainerStarted","Data":"7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8"}
Oct 30 18:58:21.049978 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:21.049849    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t" event={"ID":"4cb3fee4-1878-4b77-8ee6-ec9767749621","Type":"ContainerStarted","Data":"d29ab6ef985f3f533ebdb35d74246453188e5ec89b1210f2f498520677620daf"}
Oct 30 18:58:21.050151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:21.050132    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"
Oct 30 18:58:21.086119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:21.085309    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t" podStartSLOduration=2.085288537 podStartE2EDuration="2.085288537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:21.084361669 +0000 UTC m=+1322.800706732" watchObservedRunningTime="2026-10-30 18:58:21.085288537 +0000 UTC m=+1322.801633613"
Oct 30 18:58:21.196046 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:21.195997    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:21.624267 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:21.624231    4925 patch_prober.go:29] interesting pod/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Oct 30 18:58:21.624639 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:21.624298    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82" podUID="9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Oct 30 18:58:22.114128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:22.110606    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:22.624499 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:22.624461    4925 patch_prober.go:29] interesting pod/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Oct 30 18:58:22.624932 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:22.624530    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82" podUID="9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Oct 30 18:58:22.994122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:22.982702    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:23.264196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:23.256257    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"]
Oct 30 18:58:23.264196 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:23.256488    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerName="istio-proxy" containerID="cri-o://a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad" gracePeriod=30
Oct 30 18:58:23.285134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:23.283645    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"]
Oct 30 18:58:23.285134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:23.283884    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy" containerID="cri-o://eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c" gracePeriod=30
Oct 30 18:58:23.313133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:23.308990    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"]
Oct 30 18:58:23.313133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:23.309297    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" podUID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" containerName="istio-proxy" containerID="cri-o://6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483" gracePeriod=30
Oct 30 18:58:23.623996 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:23.623914    4925 patch_prober.go:29] interesting pod/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Oct 30 18:58:23.623996 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:23.623967    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82" podUID="9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Oct 30 18:58:23.912395 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:23.912343    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:24.150834 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:24.150759    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:24.625917 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:24.625867    4925 patch_prober.go:29] interesting pod/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Oct 30 18:58:24.626157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:24.625973    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82" podUID="9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Oct 30 18:58:24.956557 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:24.956514    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:25.624616 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:25.624579    4925 patch_prober.go:29] interesting pod/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Oct 30 18:58:25.624783 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:25.624664    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82" podUID="9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Oct 30 18:58:25.829951 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:25.827868    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:26.019634 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:26.019566    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:26.660388 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:26.660354    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"
Oct 30 18:58:26.660569 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:26.660484    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"
Oct 30 18:58:26.767685 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:26.767531    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:26.889161 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:26.881578    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:26.904402 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:26.900397    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:26.904402 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:26.903944    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 18:58:27.235909 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:27.235877    4925 generic.go:356] "Generic (PLEG): container finished" podID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerID="fc13163f5124fd3b9c9b7b77d70ad5522ee0cc6640ee149c054b1529ec077c7c" exitCode=0
Oct 30 18:58:27.236374 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:27.235954    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" event={"ID":"824748a3-c26d-4699-a479-9a5be67f3c0d","Type":"ContainerDied","Data":"fc13163f5124fd3b9c9b7b77d70ad5522ee0cc6640ee149c054b1529ec077c7c"}
Oct 30 18:58:27.236628 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:27.236609    4925 scope.go:122] "RemoveContainer" containerID="fc13163f5124fd3b9c9b7b77d70ad5522ee0cc6640ee149c054b1529ec077c7c"
Oct 30 18:58:27.238036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:27.238011    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:27.238126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:27.238075    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:27.238176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:27.238140    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:27.585109 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:27.583228    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:28.247564 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:28.247526    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" event={"ID":"824748a3-c26d-4699-a479-9a5be67f3c0d","Type":"ContainerStarted","Data":"b16e8765069999135852c2a9c8c1e26d9120c45e4d972f279b6a77fcd11b58f2"}
Oct 30 18:58:28.398568 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:28.398487    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:58:28.411545 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:28.411250    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Oct 30 18:58:28.411545 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:28.411307    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Oct 30 18:58:28.681518 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:28.681480    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:28.752836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:28.750512    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"
Oct 30 18:58:29.083876 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.083841    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:29.193049 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.190245    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:58:29.210372 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.210339    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:58:29.222137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.219203    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:58:29.278815 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.278782    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-istio-token\" (UniqueName: \"kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-istio-token\") pod \"95570521-ff70-4127-94ba-a717dc1af093\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300245    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-certs\") pod \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300303    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istiod-ca-cert\") pod \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300383    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-credential-socket\") pod \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300444    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-socket\") pod \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300496    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-envoy\") pod \"68d3f6f9-fa8f-4c43-9921-958e764d442c\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300556    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-envoy\") pod \"95570521-ff70-4127-94ba-a717dc1af093\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300591    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-certs\") pod \"95570521-ff70-4127-94ba-a717dc1af093\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300670    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-socket\") pod \"68d3f6f9-fa8f-4c43-9921-958e764d442c\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300712    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-kube-api-access-z2nqd\" (UniqueName: \"kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-kube-api-access-z2nqd\") pod \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300751    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-data\") pod \"95570521-ff70-4127-94ba-a717dc1af093\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300803    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-envoy\") pod \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300835    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-socket\") pod \"95570521-ff70-4127-94ba-a717dc1af093\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300891    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-certs\") pod \"68d3f6f9-fa8f-4c43-9921-958e764d442c\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300945    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/95570521-ff70-4127-94ba-a717dc1af093-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95570521-ff70-4127-94ba-a717dc1af093-istio-podinfo\") pod \"95570521-ff70-4127-94ba-a717dc1af093\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.300984    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-data\") pod \"68d3f6f9-fa8f-4c43-9921-958e764d442c\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.301012    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-credential-socket\") pod \"68d3f6f9-fa8f-4c43-9921-958e764d442c\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.301069    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-podinfo\") pod \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.301123    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-token\" (UniqueName: \"kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-token\") pod \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.301178    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-kube-api-access-lxs7j\" (UniqueName: \"kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-kube-api-access-lxs7j\") pod \"95570521-ff70-4127-94ba-a717dc1af093\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.301240    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-token\" (UniqueName: \"kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-token\") pod \"68d3f6f9-fa8f-4c43-9921-958e764d442c\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.301338    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-kube-api-access-7mwq2\" (UniqueName: \"kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-kube-api-access-7mwq2\") pod \"68d3f6f9-fa8f-4c43-9921-958e764d442c\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.301395    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68d3f6f9-fa8f-4c43-9921-958e764d442c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68d3f6f9-fa8f-4c43-9921-958e764d442c-istiod-ca-cert\") pod \"68d3f6f9-fa8f-4c43-9921-958e764d442c\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.301936    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95570521-ff70-4127-94ba-a717dc1af093-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95570521-ff70-4127-94ba-a717dc1af093-istiod-ca-cert\") pod \"95570521-ff70-4127-94ba-a717dc1af093\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.302176    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-data\") pod \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\" (UID: \"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.302244    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-podinfo\") pod \"68d3f6f9-fa8f-4c43-9921-958e764d442c\" (UID: \"68d3f6f9-fa8f-4c43-9921-958e764d442c\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.302280    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-credential-socket\") pod \"95570521-ff70-4127-94ba-a717dc1af093\" (UID: \"95570521-ff70-4127-94ba-a717dc1af093\") "
Oct 30 18:58:29.302641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.302424    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-certs" pod "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" (UID: "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.304198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.302803    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istiod-ca-cert" pod "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" (UID: "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:29.304198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.303088    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-credential-socket" pod "95570521-ff70-4127-94ba-a717dc1af093" (UID: "95570521-ff70-4127-94ba-a717dc1af093"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.304198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.303297    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-credential-socket" pod "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" (UID: "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.304198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.303488    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-socket" pod "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" (UID: "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.304198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.303563    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.304198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.303586    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.304198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.303602    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.304198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.303615    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.304198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.303650    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-data" pod "68d3f6f9-fa8f-4c43-9921-958e764d442c" (UID: "68d3f6f9-fa8f-4c43-9921-958e764d442c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.304198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.303852    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-credential-socket" pod "68d3f6f9-fa8f-4c43-9921-958e764d442c" (UID: "68d3f6f9-fa8f-4c43-9921-958e764d442c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.307700 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.307258    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-socket" pod "68d3f6f9-fa8f-4c43-9921-958e764d442c" (UID: "68d3f6f9-fa8f-4c43-9921-958e764d442c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.307700 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.307256    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-certs" pod "95570521-ff70-4127-94ba-a717dc1af093" (UID: "95570521-ff70-4127-94ba-a717dc1af093"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.307700 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.307389    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-envoy" pod "68d3f6f9-fa8f-4c43-9921-958e764d442c" (UID: "68d3f6f9-fa8f-4c43-9921-958e764d442c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.307700 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.307410    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-podinfo" pod "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" (UID: "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:58:29.308048 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.307679    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-certs" pod "68d3f6f9-fa8f-4c43-9921-958e764d442c" (UID: "68d3f6f9-fa8f-4c43-9921-958e764d442c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.308763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.308307    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-socket" pod "95570521-ff70-4127-94ba-a717dc1af093" (UID: "95570521-ff70-4127-94ba-a717dc1af093"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.312989 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.312825    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-data" pod "95570521-ff70-4127-94ba-a717dc1af093" (UID: "95570521-ff70-4127-94ba-a717dc1af093"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.313122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.313006    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-data" pod "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" (UID: "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.313205 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.313129    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-kube-api-access-lxs7j" pod "95570521-ff70-4127-94ba-a717dc1af093" (UID: "95570521-ff70-4127-94ba-a717dc1af093"). InnerVolumeSpecName "kube-api-access-lxs7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:29.313205 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.313188    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68d3f6f9-fa8f-4c43-9921-958e764d442c-istiod-ca-cert" pod "68d3f6f9-fa8f-4c43-9921-958e764d442c" (UID: "68d3f6f9-fa8f-4c43-9921-958e764d442c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:29.313795 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.313361    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95570521-ff70-4127-94ba-a717dc1af093-istiod-ca-cert" pod "95570521-ff70-4127-94ba-a717dc1af093" (UID: "95570521-ff70-4127-94ba-a717dc1af093"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:29.314707 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.314664    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-envoy" pod "95570521-ff70-4127-94ba-a717dc1af093" (UID: "95570521-ff70-4127-94ba-a717dc1af093"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.316283 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.315875    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-token" pod "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" (UID: "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:29.316283 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.315943    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-token" pod "68d3f6f9-fa8f-4c43-9921-958e764d442c" (UID: "68d3f6f9-fa8f-4c43-9921-958e764d442c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:29.325441 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.325388    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-kube-api-access-7mwq2" pod "68d3f6f9-fa8f-4c43-9921-958e764d442c" (UID: "68d3f6f9-fa8f-4c43-9921-958e764d442c"). InnerVolumeSpecName "kube-api-access-7mwq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.328168    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-podinfo" pod "68d3f6f9-fa8f-4c43-9921-958e764d442c" (UID: "68d3f6f9-fa8f-4c43-9921-958e764d442c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.328212    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-kube-api-access-z2nqd" pod "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" (UID: "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0"). InnerVolumeSpecName "kube-api-access-z2nqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.328211    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/95570521-ff70-4127-94ba-a717dc1af093-istio-podinfo" pod "95570521-ff70-4127-94ba-a717dc1af093" (UID: "95570521-ff70-4127-94ba-a717dc1af093"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.328246    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-envoy" pod "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" (UID: "3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.328302    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-istio-token" pod "95570521-ff70-4127-94ba-a717dc1af093" (UID: "95570521-ff70-4127-94ba-a717dc1af093"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.330738    4925 generic.go:356] "Generic (PLEG): container finished" podID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerID="a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad" exitCode=0
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.331052    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" event={"ID":"68d3f6f9-fa8f-4c43-9921-958e764d442c","Type":"ContainerDied","Data":"a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad"}
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.331228    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs" event={"ID":"68d3f6f9-fa8f-4c43-9921-958e764d442c","Type":"ContainerDied","Data":"b47beda01895ad304235a63fe4132f94de4bc1b3880c3992a656f6c935bb657c"}
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.331276    4925 scope.go:122] "RemoveContainer" containerID="a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad"
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.331607    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.333830    4925 generic.go:356] "Generic (PLEG): container finished" podID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" containerID="6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483" exitCode=0
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.333957    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" event={"ID":"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0","Type":"ContainerDied","Data":"6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483"}
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.333991    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn" event={"ID":"3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0","Type":"ContainerDied","Data":"3b8f9333290eab4e30cfcf6936a6701df2b7725e42c5c635a13cecebaed2b729"}
Oct 30 18:58:29.336389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.334061    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"
Oct 30 18:58:29.345332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.345299    4925 generic.go:356] "Generic (PLEG): container finished" podID="95570521-ff70-4127-94ba-a717dc1af093" containerID="eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c" exitCode=0
Oct 30 18:58:29.345795 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.345776    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"
Oct 30 18:58:29.346274 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.346250    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" event={"ID":"95570521-ff70-4127-94ba-a717dc1af093","Type":"ContainerDied","Data":"eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c"}
Oct 30 18:58:29.346396 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.346303    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz" event={"ID":"95570521-ff70-4127-94ba-a717dc1af093","Type":"ContainerDied","Data":"dff6567b602a03dcb5027ae91917c26304d75a9fa148608af16e04ac45d459c0"}
Oct 30 18:58:29.398883 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.398567    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Oct 30 18:58:29.399122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.398612    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Oct 30 18:58:29.399390 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.399371    4925 scope.go:122] "RemoveContainer" containerID="a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad"
Oct 30 18:58:29.399777 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:58:29.399754    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad\": container with ID starting with a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad not found: ID does not exist" containerID="a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad"
Oct 30 18:58:29.399914 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.399896    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad"} err="failed to get container status \"a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad\": rpc error: code = NotFound desc = could not find container \"a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad\": container with ID starting with a282a97d31ad4a136825420262310797bbae5fe44759179b809f5cf7d75016ad not found: ID does not exist"
Oct 30 18:58:29.400015 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.400003    4925 scope.go:122] "RemoveContainer" containerID="6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483"
Oct 30 18:58:29.405089 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405060    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405089 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405110    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405089 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405128    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405145    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405161    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2nqd\" (UniqueName: \"kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-kube-api-access-z2nqd\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405176    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405189    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405203    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95570521-ff70-4127-94ba-a717dc1af093-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405216    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405229    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95570521-ff70-4127-94ba-a717dc1af093-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405241    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405254    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68d3f6f9-fa8f-4c43-9921-958e764d442c-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405270    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405283    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405296    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxs7j\" (UniqueName: \"kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-kube-api-access-lxs7j\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405310    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405348    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mwq2\" (UniqueName: \"kubernetes.io/projected/68d3f6f9-fa8f-4c43-9921-958e764d442c-kube-api-access-7mwq2\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405364    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68d3f6f9-fa8f-4c43-9921-958e764d442c-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405377    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95570521-ff70-4127-94ba-a717dc1af093-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.405384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405390    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.406319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405404    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68d3f6f9-fa8f-4c43-9921-958e764d442c-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.406319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405417    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95570521-ff70-4127-94ba-a717dc1af093-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.406319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.405433    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:29.412561 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.412540    4925 scope.go:122] "RemoveContainer" containerID="6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483"
Oct 30 18:58:29.413049 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:58:29.413027    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483\": container with ID starting with 6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483 not found: ID does not exist" containerID="6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483"
Oct 30 18:58:29.413151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.413057    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483"} err="failed to get container status \"6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483\": rpc error: code = NotFound desc = could not find container \"6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483\": container with ID starting with 6c04003e5cc9e94987f745a42edc91261003cc78df59625afaa2432807b9b483 not found: ID does not exist"
Oct 30 18:58:29.413151 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.413074    4925 scope.go:122] "RemoveContainer" containerID="eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c"
Oct 30 18:58:29.417759 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.417732    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"]
Oct 30 18:58:29.421791 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.421764    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-sslrz"]
Oct 30 18:58:29.425411 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.425378    4925 scope.go:122] "RemoveContainer" containerID="eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c"
Oct 30 18:58:29.425766 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:58:29.425737    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c\": container with ID starting with eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c not found: ID does not exist" containerID="eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c"
Oct 30 18:58:29.425863 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.425775    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c"} err="failed to get container status \"eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c\": rpc error: code = NotFound desc = could not find container \"eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c\": container with ID starting with eba346c925d96c71dc4ef08d4e6f44ddb929e99e0beeedcecbdff4897582168c not found: ID does not exist"
Oct 30 18:58:29.440023 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.440000    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"]
Oct 30 18:58:29.445280 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.445259    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-hkbqs"]
Oct 30 18:58:29.535578 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.530195    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"]
Oct 30 18:58:29.542967 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.541790    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:29.567123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.553891    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-xxdvn"]
Oct 30 18:58:29.698405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.698359    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"]
Oct 30 18:58:29.699011 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.698979    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy" containerID="cri-o://eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1" gracePeriod=30
Oct 30 18:58:29.703704 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.703681    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"]
Oct 30 18:58:29.703935 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.703908    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" podUID="01332328-1267-4077-846f-c0553d105634" containerName="istio-proxy" containerID="cri-o://32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c" gracePeriod=30
Oct 30 18:58:29.709395 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.709148    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"]
Oct 30 18:58:29.709634 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:29.709344    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch" podUID="06139191-a477-4d2b-ab9c-30961ad65838" containerName="istio-proxy" containerID="cri-o://6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af" gracePeriod=30
Oct 30 18:58:30.045410 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.045324    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"
Oct 30 18:58:30.124548 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.124519    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"
Oct 30 18:58:30.125015 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.124995    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"
Oct 30 18:58:30.222816 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.222784    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"]
Oct 30 18:58:30.224109 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.224066    4925 cpu_manager.go:414] "containerMap: removing container" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerName="istio-proxy"
Oct 30 18:58:30.224282 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.224266    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerName="istio-proxy"
Oct 30 18:58:30.224394 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.224383    4925 cpu_manager.go:414] "containerMap: removing container" podUID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" containerName="istio-proxy"
Oct 30 18:58:30.224472 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.224462    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" containerName="istio-proxy"
Oct 30 18:58:30.224602 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.224588    4925 cpu_manager.go:414] "containerMap: removing container" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy"
Oct 30 18:58:30.224704 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.224693    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy"
Oct 30 18:58:30.225322 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.225032    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" containerName="istio-proxy"
Oct 30 18:58:30.225467 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.225452    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" containerName="istio-proxy"
Oct 30 18:58:30.225559 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.225549    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="95570521-ff70-4127-94ba-a717dc1af093" containerName="istio-proxy"
Oct 30 18:58:30.233042 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.232991    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.236283 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.236265    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"]
Oct 30 18:58:30.237982 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.237932    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-f778f5-serving-cert\""
Oct 30 18:58:30.330815 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.330722    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f00d2ad1-ff8f-481c-82cf-74f20531ddde-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.330815 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.330807    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.331245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.331089    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-metrics-certs\") pod \"maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.331245 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.331206    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkrb5\" (UniqueName: \"kubernetes.io/projected/f00d2ad1-ff8f-481c-82cf-74f20531ddde-kube-api-access-nkrb5\") pod \"maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.389499 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.389464    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:30.397963 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.397937    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Oct 30 18:58:30.398127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.397984    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Oct 30 18:58:30.429775 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.429744    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0" path="/var/lib/kubelet/pods/3b9a3b20-f4ae-40b0-8bc1-4c51bd3687e0/volumes"
Oct 30 18:58:30.430250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.430234    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68d3f6f9-fa8f-4c43-9921-958e764d442c" path="/var/lib/kubelet/pods/68d3f6f9-fa8f-4c43-9921-958e764d442c/volumes"
Oct 30 18:58:30.430692 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.430677    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95570521-ff70-4127-94ba-a717dc1af093" path="/var/lib/kubelet/pods/95570521-ff70-4127-94ba-a717dc1af093/volumes"
Oct 30 18:58:30.432544 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.432520    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-metrics-certs\") pod \"maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.432693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.432569    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkrb5\" (UniqueName: \"kubernetes.io/projected/f00d2ad1-ff8f-481c-82cf-74f20531ddde-kube-api-access-nkrb5\") pod \"maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.432693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.432623    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f00d2ad1-ff8f-481c-82cf-74f20531ddde-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.432810 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.432698    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.433451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.433397    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f00d2ad1-ff8f-481c-82cf-74f20531ddde-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.435371 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.435351    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.435452 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.435385    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-metrics-certs\") pod \"maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.440927 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.440900    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkrb5\" (UniqueName: \"kubernetes.io/projected/f00d2ad1-ff8f-481c-82cf-74f20531ddde-kube-api-access-nkrb5\") pod \"maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.555545 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.555508    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:30.584490 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:30.584337    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:31.226125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:31.221849    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"]
Oct 30 18:58:31.255904 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:31.255153    4925 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:58:31.395059 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:31.395025    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw" event={"ID":"f00d2ad1-ff8f-481c-82cf-74f20531ddde","Type":"ContainerStarted","Data":"77f3f98fd59308b34be15c021bab15e1fe6c84bf598944f83295e911b874ca06"}
Oct 30 18:58:31.398340 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:31.398313    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Oct 30 18:58:31.398483 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:31.398362    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Oct 30 18:58:31.498187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:31.489820    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:32.136814 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:32.136732    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"
Oct 30 18:58:32.218318 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:32.218285    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:32.358844 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:32.357195    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:58:32.397788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:32.397701    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Oct 30 18:58:32.398361 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:32.398329    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Oct 30 18:58:32.410206 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:32.410171    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw" event={"ID":"f00d2ad1-ff8f-481c-82cf-74f20531ddde","Type":"ContainerStarted","Data":"9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395"}
Oct 30 18:58:32.410406 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:32.410310    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:32.429091 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:32.429049    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw" podStartSLOduration=1.958607357 podStartE2EDuration="2.429036463s" totalImagesPullingTime="470.429106ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:32.425836979 +0000 UTC m=+1334.142182057" watchObservedRunningTime="2026-10-30 18:58:32.429036463 +0000 UTC m=+1334.145381539"
Oct 30 18:58:33.098124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:33.097705    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:33.403623 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:33.403539    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused" start-of-body=
Oct 30 18:58:33.404232 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:33.404201    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.129:15021/healthz/ready\": dial tcp 10.132.0.129:15021: connect: connection refused"
Oct 30 18:58:33.870122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:33.857331    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:34.403229 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:34.403199    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:58:34.404640 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:34.404612    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 18:58:34.693121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:34.692443    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:35.031054 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.031028    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:58:35.050511 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.050479    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/06139191-a477-4d2b-ab9c-30961ad65838-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06139191-a477-4d2b-ab9c-30961ad65838-istio-podinfo\") pod \"06139191-a477-4d2b-ab9c-30961ad65838\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") "
Oct 30 18:58:35.050658 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.050530    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-credential-socket\") pod \"06139191-a477-4d2b-ab9c-30961ad65838\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") "
Oct 30 18:58:35.050658 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.050561    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-data\") pod \"06139191-a477-4d2b-ab9c-30961ad65838\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") "
Oct 30 18:58:35.050658 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.050634    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-socket\") pod \"06139191-a477-4d2b-ab9c-30961ad65838\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") "
Oct 30 18:58:35.050869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.050674    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-certs\") pod \"06139191-a477-4d2b-ab9c-30961ad65838\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") "
Oct 30 18:58:35.050869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.050734    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-istio-token\" (UniqueName: \"kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-istio-token\") pod \"06139191-a477-4d2b-ab9c-30961ad65838\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") "
Oct 30 18:58:35.050869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.050764    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06139191-a477-4d2b-ab9c-30961ad65838-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06139191-a477-4d2b-ab9c-30961ad65838-istiod-ca-cert\") pod \"06139191-a477-4d2b-ab9c-30961ad65838\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") "
Oct 30 18:58:35.050869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.050809    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-credential-socket" pod "06139191-a477-4d2b-ab9c-30961ad65838" (UID: "06139191-a477-4d2b-ab9c-30961ad65838"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.050869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.050842    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-kube-api-access-nzghv\" (UniqueName: \"kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-kube-api-access-nzghv\") pod \"06139191-a477-4d2b-ab9c-30961ad65838\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") "
Oct 30 18:58:35.051062 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.050927    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-data" pod "06139191-a477-4d2b-ab9c-30961ad65838" (UID: "06139191-a477-4d2b-ab9c-30961ad65838"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.051062 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.050969    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-envoy\") pod \"06139191-a477-4d2b-ab9c-30961ad65838\" (UID: \"06139191-a477-4d2b-ab9c-30961ad65838\") "
Oct 30 18:58:35.051158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.051136    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-socket" pod "06139191-a477-4d2b-ab9c-30961ad65838" (UID: "06139191-a477-4d2b-ab9c-30961ad65838"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.051452 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.051423    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06139191-a477-4d2b-ab9c-30961ad65838-istiod-ca-cert" pod "06139191-a477-4d2b-ab9c-30961ad65838" (UID: "06139191-a477-4d2b-ab9c-30961ad65838"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:35.051796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.051732    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.051796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.051763    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.051796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.051778    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.051796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.051791    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06139191-a477-4d2b-ab9c-30961ad65838-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.053496 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.053444    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-certs" pod "06139191-a477-4d2b-ab9c-30961ad65838" (UID: "06139191-a477-4d2b-ab9c-30961ad65838"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.059445 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.059336    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-istio-token" pod "06139191-a477-4d2b-ab9c-30961ad65838" (UID: "06139191-a477-4d2b-ab9c-30961ad65838"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:35.059445 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.059357    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-envoy" pod "06139191-a477-4d2b-ab9c-30961ad65838" (UID: "06139191-a477-4d2b-ab9c-30961ad65838"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.059445 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.059405    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-kube-api-access-nzghv" pod "06139191-a477-4d2b-ab9c-30961ad65838" (UID: "06139191-a477-4d2b-ab9c-30961ad65838"). InnerVolumeSpecName "kube-api-access-nzghv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:35.059622 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.059452    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06139191-a477-4d2b-ab9c-30961ad65838-istio-podinfo" pod "06139191-a477-4d2b-ab9c-30961ad65838" (UID: "06139191-a477-4d2b-ab9c-30961ad65838"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:58:35.086766 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.086738    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:58:35.152764 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.152719    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-kube-api-access-s5rm7\" (UniqueName: \"kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-kube-api-access-s5rm7\") pod \"01332328-1267-4077-846f-c0553d105634\" (UID: \"01332328-1267-4077-846f-c0553d105634\") "
Oct 30 18:58:35.152944 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.152784    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-data\") pod \"01332328-1267-4077-846f-c0553d105634\" (UID: \"01332328-1267-4077-846f-c0553d105634\") "
Oct 30 18:58:35.152944 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.152896    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/01332328-1267-4077-846f-c0553d105634-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01332328-1267-4077-846f-c0553d105634-istio-podinfo\") pod \"01332328-1267-4077-846f-c0553d105634\" (UID: \"01332328-1267-4077-846f-c0553d105634\") "
Oct 30 18:58:35.153158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.153061    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01332328-1267-4077-846f-c0553d105634-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01332328-1267-4077-846f-c0553d105634-istiod-ca-cert\") pod \"01332328-1267-4077-846f-c0553d105634\" (UID: \"01332328-1267-4077-846f-c0553d105634\") "
Oct 30 18:58:35.153158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.153151    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-istio-token\" (UniqueName: \"kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-istio-token\") pod \"01332328-1267-4077-846f-c0553d105634\" (UID: \"01332328-1267-4077-846f-c0553d105634\") "
Oct 30 18:58:35.153292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.153185    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-credential-socket\") pod \"01332328-1267-4077-846f-c0553d105634\" (UID: \"01332328-1267-4077-846f-c0553d105634\") "
Oct 30 18:58:35.153292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.153214    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-envoy\") pod \"01332328-1267-4077-846f-c0553d105634\" (UID: \"01332328-1267-4077-846f-c0553d105634\") "
Oct 30 18:58:35.153292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.153272    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-certs\") pod \"01332328-1267-4077-846f-c0553d105634\" (UID: \"01332328-1267-4077-846f-c0553d105634\") "
Oct 30 18:58:35.153451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.153306    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-data" pod "01332328-1267-4077-846f-c0553d105634" (UID: "01332328-1267-4077-846f-c0553d105634"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.153451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.153319    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-socket\") pod \"01332328-1267-4077-846f-c0553d105634\" (UID: \"01332328-1267-4077-846f-c0553d105634\") "
Oct 30 18:58:35.154130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.154006    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.154130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.154031    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.154130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.154047    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.154130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.154061    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzghv\" (UniqueName: \"kubernetes.io/projected/06139191-a477-4d2b-ab9c-30961ad65838-kube-api-access-nzghv\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.154130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.154075    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06139191-a477-4d2b-ab9c-30961ad65838-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.154452 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.154136    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06139191-a477-4d2b-ab9c-30961ad65838-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.154452 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.154341    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-socket" pod "01332328-1267-4077-846f-c0553d105634" (UID: "01332328-1267-4077-846f-c0553d105634"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.154554 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.154520    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-certs" pod "01332328-1267-4077-846f-c0553d105634" (UID: "01332328-1267-4077-846f-c0553d105634"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.154723 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.154676    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-credential-socket" pod "01332328-1267-4077-846f-c0553d105634" (UID: "01332328-1267-4077-846f-c0553d105634"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.154821 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.154800    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01332328-1267-4077-846f-c0553d105634-istiod-ca-cert" pod "01332328-1267-4077-846f-c0553d105634" (UID: "01332328-1267-4077-846f-c0553d105634"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:35.165292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.156210    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-istio-token" pod "01332328-1267-4077-846f-c0553d105634" (UID: "01332328-1267-4077-846f-c0553d105634"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:35.165292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.159831    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/01332328-1267-4077-846f-c0553d105634-istio-podinfo" pod "01332328-1267-4077-846f-c0553d105634" (UID: "01332328-1267-4077-846f-c0553d105634"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:58:35.165292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.160008    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-kube-api-access-s5rm7" pod "01332328-1267-4077-846f-c0553d105634" (UID: "01332328-1267-4077-846f-c0553d105634"). InnerVolumeSpecName "kube-api-access-s5rm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:35.165292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.162116    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-envoy" pod "01332328-1267-4077-846f-c0553d105634" (UID: "01332328-1267-4077-846f-c0553d105634"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.255617 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.255566    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.255617 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.255599    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.255617 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.255609    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.255793 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.255619    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01332328-1267-4077-846f-c0553d105634-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.255793 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.255630    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5rm7\" (UniqueName: \"kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-kube-api-access-s5rm7\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.255793 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.255646    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01332328-1267-4077-846f-c0553d105634-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.255793 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.255655    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01332328-1267-4077-846f-c0553d105634-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.255793 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.255664    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01332328-1267-4077-846f-c0553d105634-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.428316 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.428289    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:58:35.440573 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.440547    4925 generic.go:356] "Generic (PLEG): container finished" podID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerID="eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1" exitCode=0
Oct 30 18:58:35.440720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.440590    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"
Oct 30 18:58:35.440720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.440629    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" event={"ID":"12ddc094-da4b-432c-a802-0b21b67e84b8","Type":"ContainerDied","Data":"eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1"}
Oct 30 18:58:35.440720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.440686    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p" event={"ID":"12ddc094-da4b-432c-a802-0b21b67e84b8","Type":"ContainerDied","Data":"2011b413f32480d605f75c6c96a45e28a722c2ad71777d6fd9f62dcf5ec67fc0"}
Oct 30 18:58:35.440720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.440715    4925 scope.go:122] "RemoveContainer" containerID="eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1"
Oct 30 18:58:35.442153 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.442129    4925 generic.go:356] "Generic (PLEG): container finished" podID="06139191-a477-4d2b-ab9c-30961ad65838" containerID="6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af" exitCode=0
Oct 30 18:58:35.442270 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.442170    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"
Oct 30 18:58:35.442270 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.442222    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch" event={"ID":"06139191-a477-4d2b-ab9c-30961ad65838","Type":"ContainerDied","Data":"6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af"}
Oct 30 18:58:35.442389 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.442279    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch" event={"ID":"06139191-a477-4d2b-ab9c-30961ad65838","Type":"ContainerDied","Data":"481c40ac00afd53f96676ad781f3dce24182db271c6a53a75d017f3722af1529"}
Oct 30 18:58:35.443887 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.443856    4925 generic.go:356] "Generic (PLEG): container finished" podID="01332328-1267-4077-846f-c0553d105634" containerID="32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c" exitCode=0
Oct 30 18:58:35.443997 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.443897    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"
Oct 30 18:58:35.443997 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.443931    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" event={"ID":"01332328-1267-4077-846f-c0553d105634","Type":"ContainerDied","Data":"32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c"}
Oct 30 18:58:35.443997 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.443969    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs" event={"ID":"01332328-1267-4077-846f-c0553d105634","Type":"ContainerDied","Data":"88f1a182a87ced8c2a4ddc657446e65ef5a904c0916476717fcb99cbaaf29b69"}
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.458330    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-certs\") pod \"12ddc094-da4b-432c-a802-0b21b67e84b8\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") "
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.458368    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-socket\") pod \"12ddc094-da4b-432c-a802-0b21b67e84b8\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") "
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.458462    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-token\" (UniqueName: \"kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-token\") pod \"12ddc094-da4b-432c-a802-0b21b67e84b8\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") "
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.458517    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-envoy\") pod \"12ddc094-da4b-432c-a802-0b21b67e84b8\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") "
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.458649    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12ddc094-da4b-432c-a802-0b21b67e84b8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12ddc094-da4b-432c-a802-0b21b67e84b8-istiod-ca-cert\") pod \"12ddc094-da4b-432c-a802-0b21b67e84b8\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") "
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.458687    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-credential-socket\") pod \"12ddc094-da4b-432c-a802-0b21b67e84b8\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") "
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.458736    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-data\") pod \"12ddc094-da4b-432c-a802-0b21b67e84b8\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") "
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.458792    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-podinfo\") pod \"12ddc094-da4b-432c-a802-0b21b67e84b8\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") "
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.458833    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-kube-api-access-kkmrw\" (UniqueName: \"kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-kube-api-access-kkmrw\") pod \"12ddc094-da4b-432c-a802-0b21b67e84b8\" (UID: \"12ddc094-da4b-432c-a802-0b21b67e84b8\") "
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.460360    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-socket" pod "12ddc094-da4b-432c-a802-0b21b67e84b8" (UID: "12ddc094-da4b-432c-a802-0b21b67e84b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.460381    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12ddc094-da4b-432c-a802-0b21b67e84b8-istiod-ca-cert" pod "12ddc094-da4b-432c-a802-0b21b67e84b8" (UID: "12ddc094-da4b-432c-a802-0b21b67e84b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.460561    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-certs" pod "12ddc094-da4b-432c-a802-0b21b67e84b8" (UID: "12ddc094-da4b-432c-a802-0b21b67e84b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.461381    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-credential-socket" pod "12ddc094-da4b-432c-a802-0b21b67e84b8" (UID: "12ddc094-da4b-432c-a802-0b21b67e84b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.463485 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.461540    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-data" pod "12ddc094-da4b-432c-a802-0b21b67e84b8" (UID: "12ddc094-da4b-432c-a802-0b21b67e84b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.464166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.463719    4925 scope.go:122] "RemoveContainer" containerID="eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1"
Oct 30 18:58:35.464273 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:58:35.464238    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1\": container with ID starting with eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1 not found: ID does not exist" containerID="eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1"
Oct 30 18:58:35.464395 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.464282    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1"} err="failed to get container status \"eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1\": rpc error: code = NotFound desc = could not find container \"eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1\": container with ID starting with eac877c3829a00a7fa454eff4c7ca5b49807623b0c59c5918f86f596b8a909c1 not found: ID does not exist"
Oct 30 18:58:35.464395 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.464310    4925 scope.go:122] "RemoveContainer" containerID="6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af"
Oct 30 18:58:35.475561 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.475519    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-token" pod "12ddc094-da4b-432c-a802-0b21b67e84b8" (UID: "12ddc094-da4b-432c-a802-0b21b67e84b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:35.475796 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.475776    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-podinfo" pod "12ddc094-da4b-432c-a802-0b21b67e84b8" (UID: "12ddc094-da4b-432c-a802-0b21b67e84b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:58:35.476061 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.476034    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-kube-api-access-kkmrw" pod "12ddc094-da4b-432c-a802-0b21b67e84b8" (UID: "12ddc094-da4b-432c-a802-0b21b67e84b8"). InnerVolumeSpecName "kube-api-access-kkmrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:35.476397 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.476369    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-envoy" pod "12ddc094-da4b-432c-a802-0b21b67e84b8" (UID: "12ddc094-da4b-432c-a802-0b21b67e84b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:35.481270 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.480895    4925 scope.go:122] "RemoveContainer" containerID="6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af"
Oct 30 18:58:35.481747 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:58:35.481426    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af\": container with ID starting with 6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af not found: ID does not exist" containerID="6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af"
Oct 30 18:58:35.481747 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.481463    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af"} err="failed to get container status \"6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af\": rpc error: code = NotFound desc = could not find container \"6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af\": container with ID starting with 6dcbe487c467c941f8b4a38f2585d760e538dd53ae2579fce7990c1d28aec2af not found: ID does not exist"
Oct 30 18:58:35.481747 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.481485    4925 scope.go:122] "RemoveContainer" containerID="32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c"
Oct 30 18:58:35.489889 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.489847    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"]
Oct 30 18:58:35.493240 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.493086    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-d6xvs"]
Oct 30 18:58:35.504393 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.501053    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"]
Oct 30 18:58:35.505583 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.505569    4925 scope.go:122] "RemoveContainer" containerID="32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c"
Oct 30 18:58:35.506492 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.506439    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8ljch"]
Oct 30 18:58:35.506586 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:58:35.506545    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c\": container with ID starting with 32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c not found: ID does not exist" containerID="32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c"
Oct 30 18:58:35.506660 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.506574    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c"} err="failed to get container status \"32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c\": rpc error: code = NotFound desc = could not find container \"32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c\": container with ID starting with 32c9cc8482e304cfc20c2c69505667ebc84c4d7e24b37d7542cceca67b544b2c not found: ID does not exist"
Oct 30 18:58:35.546731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.546705    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:35.577133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.573184    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.577133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.573223    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.577133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.573247    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.577133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.573261    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.577133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.573275    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12ddc094-da4b-432c-a802-0b21b67e84b8-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.577133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.573288    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.577133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.573330    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.577133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.573349    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12ddc094-da4b-432c-a802-0b21b67e84b8-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.577133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.573491    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkmrw\" (UniqueName: \"kubernetes.io/projected/12ddc094-da4b-432c-a802-0b21b67e84b8-kube-api-access-kkmrw\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:35.765076 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.764995    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"]
Oct 30 18:58:35.769433 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:35.769404    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-fjf7p"]
Oct 30 18:58:36.270843 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:36.270797    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:36.426592 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:36.426555    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01332328-1267-4077-846f-c0553d105634" path="/var/lib/kubelet/pods/01332328-1267-4077-846f-c0553d105634/volumes"
Oct 30 18:58:36.427269 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:36.427241    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06139191-a477-4d2b-ab9c-30961ad65838" path="/var/lib/kubelet/pods/06139191-a477-4d2b-ab9c-30961ad65838/volumes"
Oct 30 18:58:36.427926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:36.427902    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" path="/var/lib/kubelet/pods/12ddc094-da4b-432c-a802-0b21b67e84b8/volumes"
Oct 30 18:58:37.056124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:37.055314    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:37.708731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:37.705718    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:38.425885 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:38.425861    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:58:38.760945 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:38.760856    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:38.950027 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:38.949995    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:39.561138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:39.559733    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:40.104879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.104846    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:40.293039 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.293006    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:40.414421 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.414384    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"]
Oct 30 18:58:40.415140 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.415123    4925 cpu_manager.go:414] "containerMap: removing container" podUID="06139191-a477-4d2b-ab9c-30961ad65838" containerName="istio-proxy"
Oct 30 18:58:40.415140 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.415141    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06139191-a477-4d2b-ab9c-30961ad65838" containerName="istio-proxy"
Oct 30 18:58:40.415290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.415165    4925 cpu_manager.go:414] "containerMap: removing container" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy"
Oct 30 18:58:40.415290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.415170    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy"
Oct 30 18:58:40.415290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.415191    4925 cpu_manager.go:414] "containerMap: removing container" podUID="01332328-1267-4077-846f-c0553d105634" containerName="istio-proxy"
Oct 30 18:58:40.415290 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.415196    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01332328-1267-4077-846f-c0553d105634" containerName="istio-proxy"
Oct 30 18:58:40.415501 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.415333    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="06139191-a477-4d2b-ab9c-30961ad65838" containerName="istio-proxy"
Oct 30 18:58:40.415501 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.415345    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="12ddc094-da4b-432c-a802-0b21b67e84b8" containerName="istio-proxy"
Oct 30 18:58:40.415501 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.415353    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="01332328-1267-4077-846f-c0553d105634" containerName="istio-proxy"
Oct 30 18:58:40.420885 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.420842    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.426499 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.426479    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-10d367-openshift-default-dockercfg-dmbcf\""
Oct 30 18:58:40.447301 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.447274    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"]
Oct 30 18:58:40.548729 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.548697    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:40.571272 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.571246    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-envoy\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.571675 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.571362    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-credential-socket\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.571675 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.571400    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-podinfo\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.571675 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.571516    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41a16b6-cb27-412e-90a6-5374b69b8a76-istiod-ca-cert\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.571675 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.571551    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-data\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.571675 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.571652    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxhr5\" (UniqueName: \"kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-kube-api-access-vxhr5\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.571862 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.571773    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-certs\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.571925 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.571897    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-token\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.571983 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.571962    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-socket\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.677323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.677229    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-envoy\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.677629 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.677602    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-credential-socket\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.679431 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.679409    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-podinfo\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.689129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.680018    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41a16b6-cb27-412e-90a6-5374b69b8a76-istiod-ca-cert\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.690011 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.689986    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-data\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.690754 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.690735    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxhr5\" (UniqueName: \"kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-kube-api-access-vxhr5\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.691405 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.691382    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-certs\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.691940 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.691920    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-token\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.704378 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.704350    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-socket\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.705163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.683780    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41a16b6-cb27-412e-90a6-5374b69b8a76-istiod-ca-cert\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.705328 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.689847    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-podinfo\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.719116 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.691768    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-certs\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.720875 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.679336    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-credential-socket\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.721051 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.690628    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-data\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.721187 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.704834    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-socket\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.721318 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.717485    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-envoy\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.721424 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.720783    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxhr5\" (UniqueName: \"kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-kube-api-access-vxhr5\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.721813 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.721782    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-token\") pod \"e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") " pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:40.737089 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.737052    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:40.748669 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:40.748628    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:41.170228 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:41.119037    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"]
Oct 30 18:58:41.208700 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:41.201775    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:41.208700 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:41.202981    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:41.208700 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:41.203111    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:41.384122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:41.380314    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:41.509172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:41.508583    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" event={"ID":"e41a16b6-cb27-412e-90a6-5374b69b8a76","Type":"ContainerStarted","Data":"5cc8988293ddfd81bf7c75ae9876ae7ee4971ae29a78bf43966d459bb3e49f10"}
Oct 30 18:58:41.509172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:41.508644    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" event={"ID":"e41a16b6-cb27-412e-90a6-5374b69b8a76","Type":"ContainerStarted","Data":"afebe8080be7d790512e78b43163c3d6990adcfc76c1b078e63658a54616976d"}
Oct 30 18:58:41.528577 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:41.528521    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" podStartSLOduration=1.528504985 podStartE2EDuration="1.528504985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:41.527705116 +0000 UTC m=+1343.244050192" watchObservedRunningTime="2026-10-30 18:58:41.528504985 +0000 UTC m=+1343.244850059"
Oct 30 18:58:41.749833 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:41.749797    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:41.944629 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:41.944597    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"]
Oct 30 18:58:41.963620 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:41.963584    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"]
Oct 30 18:58:42.109428 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:42.109389    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:42.294743 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:42.294659    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:42.636478 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:42.636403    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:42.768782 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:42.768451    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:43.454759 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:58:43.454709    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:58:43.525874 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:43.525846    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:43.526926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:43.526905    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:58:43.911178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:43.910880    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:44.127660 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:44.127626    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:45.406119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.382005    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"]
Oct 30 18:58:45.563124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.548677    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"
Oct 30 18:58:45.608117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.605069    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-10d367\""
Oct 30 18:58:45.644133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.610172    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"]
Oct 30 18:58:45.729067 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.691572    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqtsp\" (UniqueName: \"kubernetes.io/projected/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-kube-api-access-pqtsp\") pod \"payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q\" (UID: \"89911b2c-ec95-4aa1-b6cc-5c712d5cabae\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"
Oct 30 18:58:45.729067 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.691857    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q\" (UID: \"89911b2c-ec95-4aa1-b6cc-5c712d5cabae\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"
Oct 30 18:58:45.732134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.732107    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"]
Oct 30 18:58:45.840910 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.774191    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"
Oct 30 18:58:45.840910 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.775385    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"]
Oct 30 18:58:45.848881 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.842203    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-10d367-dockercfg-jgftb\""
Oct 30 18:58:45.905117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.898480    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:45.925071 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.924725    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q\" (UID: \"89911b2c-ec95-4aa1-b6cc-5c712d5cabae\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"
Oct 30 18:58:45.925071 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.924847    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqtsp\" (UniqueName: \"kubernetes.io/projected/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-kube-api-access-pqtsp\") pod \"payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q\" (UID: \"89911b2c-ec95-4aa1-b6cc-5c712d5cabae\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"
Oct 30 18:58:45.927599 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:45.927558    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q\" (UID: \"89911b2c-ec95-4aa1-b6cc-5c712d5cabae\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"
Oct 30 18:58:46.014189 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.006145    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqtsp\" (UniqueName: \"kubernetes.io/projected/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-kube-api-access-pqtsp\") pod \"payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q\" (UID: \"89911b2c-ec95-4aa1-b6cc-5c712d5cabae\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"
Oct 30 18:58:46.040262 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.039946    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fae6cb79-cee1-48f6-aff1-29f3b3bff735-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z\" (UID: \"fae6cb79-cee1-48f6-aff1-29f3b3bff735\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"
Oct 30 18:58:46.040262 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.040224    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bntj\" (UniqueName: \"kubernetes.io/projected/fae6cb79-cee1-48f6-aff1-29f3b3bff735-kube-api-access-8bntj\") pod \"payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z\" (UID: \"fae6cb79-cee1-48f6-aff1-29f3b3bff735\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"
Oct 30 18:58:46.077577 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.077548    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"
Oct 30 18:58:46.203126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.201238    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:46.203126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.202020    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fae6cb79-cee1-48f6-aff1-29f3b3bff735-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z\" (UID: \"fae6cb79-cee1-48f6-aff1-29f3b3bff735\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"
Oct 30 18:58:46.203126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.202283    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bntj\" (UniqueName: \"kubernetes.io/projected/fae6cb79-cee1-48f6-aff1-29f3b3bff735-kube-api-access-8bntj\") pod \"payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z\" (UID: \"fae6cb79-cee1-48f6-aff1-29f3b3bff735\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"
Oct 30 18:58:46.203502 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.203472    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fae6cb79-cee1-48f6-aff1-29f3b3bff735-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z\" (UID: \"fae6cb79-cee1-48f6-aff1-29f3b3bff735\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"
Oct 30 18:58:46.263717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.263665    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:46.353222 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.268782    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bntj\" (UniqueName: \"kubernetes.io/projected/fae6cb79-cee1-48f6-aff1-29f3b3bff735-kube-api-access-8bntj\") pod \"payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z\" (UID: \"fae6cb79-cee1-48f6-aff1-29f3b3bff735\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"
Oct 30 18:58:46.466608 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.466574    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"
Oct 30 18:58:46.849121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.840836    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:46.938580 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.938480    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:46.970238 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.970184    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf" podUID="6ac6b3d6-6972-4721-af33-90a13e518e4e" containerName="payload-pre-processing" containerID="cri-o://98cec756b152b16181640fe95b3aad1e7d0d276971579e50b58d7408dfd0c452" gracePeriod=25
Oct 30 18:58:46.970632 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:46.970602    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg" podUID="73b71a46-e413-45f2-a484-415344e068d3" containerName="payload-processing" containerID="cri-o://efb7619b2cb6869a2e0402fe1c5892749f3f58e51a0c9f1a15625726b72ef065" gracePeriod=25
Oct 30 18:58:47.879118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:47.865586    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"]
Oct 30 18:58:47.901675 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:47.901641    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"]
Oct 30 18:58:48.212124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:48.211212    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm" podUID="47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.128:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:48.336133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:48.320643    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:48.900182 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:48.889882    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm" podUID="47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.128:8000/health\": context deadline exceeded"
Oct 30 18:58:48.900182 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:48.890349    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:48.950306 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:48.948262    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:48.950306 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:48.949766    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 18:58:49.020497 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.020457    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:49.151020 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.108037    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:49.251627 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.251577    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm" podUID="47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.128:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:49.252811 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.252781    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/health\": context deadline exceeded"
Oct 30 18:58:49.350416 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.350379    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 18:58:49.441363 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.433232    4925 generic.go:356] "Generic (PLEG): container finished" podID="6ac6b3d6-6972-4721-af33-90a13e518e4e" containerID="98cec756b152b16181640fe95b3aad1e7d0d276971579e50b58d7408dfd0c452" exitCode=0
Oct 30 18:58:49.620137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.619606    4925 generic.go:356] "Generic (PLEG): container finished" podID="73b71a46-e413-45f2-a484-415344e068d3" containerID="efb7619b2cb6869a2e0402fe1c5892749f3f58e51a0c9f1a15625726b72ef065" exitCode=0
Oct 30 18:58:49.630157 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.629348    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"
Oct 30 18:58:49.652388 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:58:49.649474    4925 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.235s"
Oct 30 18:58:49.652388 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.649515    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z" event={"ID":"fae6cb79-cee1-48f6-aff1-29f3b3bff735","Type":"ContainerStarted","Data":"faf236d3d5efa5259b108575f39daa408252a650fdd376edc7fe70cbb2c38e18"}
Oct 30 18:58:49.652388 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.649770    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf" event={"ID":"6ac6b3d6-6972-4721-af33-90a13e518e4e","Type":"ContainerDied","Data":"98cec756b152b16181640fe95b3aad1e7d0d276971579e50b58d7408dfd0c452"}
Oct 30 18:58:49.652388 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.649822    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q" event={"ID":"89911b2c-ec95-4aa1-b6cc-5c712d5cabae","Type":"ContainerStarted","Data":"9f05766928da13cdb04dcf50e75196dc1227d66905df658c6281c34b6df14402"}
Oct 30 18:58:49.653008 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.652989    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:58:49.653189 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.653171    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg" event={"ID":"73b71a46-e413-45f2-a484-415344e068d3","Type":"ContainerDied","Data":"efb7619b2cb6869a2e0402fe1c5892749f3f58e51a0c9f1a15625726b72ef065"}
Oct 30 18:58:49.653829 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.653812    4925 scope.go:122] "RemoveContainer" containerID="efb7619b2cb6869a2e0402fe1c5892749f3f58e51a0c9f1a15625726b72ef065"
Oct 30 18:58:49.809792 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.809756    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73b71a46-e413-45f2-a484-415344e068d3-plugins-config-volume" pod "73b71a46-e413-45f2-a484-415344e068d3" (UID: "73b71a46-e413-45f2-a484-415344e068d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:49.810061 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.810037    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73b71a46-e413-45f2-a484-415344e068d3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b71a46-e413-45f2-a484-415344e068d3-plugins-config-volume\") pod \"73b71a46-e413-45f2-a484-415344e068d3\" (UID: \"73b71a46-e413-45f2-a484-415344e068d3\") "
Oct 30 18:58:49.811154 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.811132    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73b71a46-e413-45f2-a484-415344e068d3-kube-api-access-r2nnc\" (UniqueName: \"kubernetes.io/projected/73b71a46-e413-45f2-a484-415344e068d3-kube-api-access-r2nnc\") pod \"73b71a46-e413-45f2-a484-415344e068d3\" (UID: \"73b71a46-e413-45f2-a484-415344e068d3\") "
Oct 30 18:58:49.828146 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.821024    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b71a46-e413-45f2-a484-415344e068d3-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:49.847706 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.847667    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73b71a46-e413-45f2-a484-415344e068d3-kube-api-access-r2nnc" pod "73b71a46-e413-45f2-a484-415344e068d3" (UID: "73b71a46-e413-45f2-a484-415344e068d3"). InnerVolumeSpecName "kube-api-access-r2nnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:49.897300 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.897251    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:49.922141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.920612    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:58:49.926130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:49.923830    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2nnc\" (UniqueName: \"kubernetes.io/projected/73b71a46-e413-45f2-a484-415344e068d3-kube-api-access-r2nnc\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:50.022888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.020881    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:50.022888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.021581    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:58:50.228330 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.224762    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:50.392506 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.392440    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm" podUID="47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.128:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:50.544121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.482701    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:58:50.544121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.526221    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:50.609140 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.606305    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"
Oct 30 18:58:50.709913 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.709877    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ac6b3d6-6972-4721-af33-90a13e518e4e-kube-api-access-fjmb2\" (UniqueName: \"kubernetes.io/projected/6ac6b3d6-6972-4721-af33-90a13e518e4e-kube-api-access-fjmb2\") pod \"6ac6b3d6-6972-4721-af33-90a13e518e4e\" (UID: \"6ac6b3d6-6972-4721-af33-90a13e518e4e\") "
Oct 30 18:58:50.732662 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.732434    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac6b3d6-6972-4721-af33-90a13e518e4e-kube-api-access-fjmb2" pod "6ac6b3d6-6972-4721-af33-90a13e518e4e" (UID: "6ac6b3d6-6972-4721-af33-90a13e518e4e"). InnerVolumeSpecName "kube-api-access-fjmb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:50.751976 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.751946    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 18:58:50.795024 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.794990    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg" event={"ID":"73b71a46-e413-45f2-a484-415344e068d3","Type":"ContainerDied","Data":"61dcb3aa2b57e7d51cf6f2720a0241b3d881af0408e6f6c9d6e2dd11107ecd0d"}
Oct 30 18:58:50.795384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.795367    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"
Oct 30 18:58:50.814004 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.811386    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ac6b3d6-6972-4721-af33-90a13e518e4e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ac6b3d6-6972-4721-af33-90a13e518e4e-plugins-config-volume\") pod \"6ac6b3d6-6972-4721-af33-90a13e518e4e\" (UID: \"6ac6b3d6-6972-4721-af33-90a13e518e4e\") "
Oct 30 18:58:50.816756 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.814281    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjmb2\" (UniqueName: \"kubernetes.io/projected/6ac6b3d6-6972-4721-af33-90a13e518e4e-kube-api-access-fjmb2\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:50.831615 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.831583    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z" event={"ID":"fae6cb79-cee1-48f6-aff1-29f3b3bff735","Type":"ContainerStarted","Data":"eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9"}
Oct 30 18:58:50.832652 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.832630    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"
Oct 30 18:58:50.838847 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.838821    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ac6b3d6-6972-4721-af33-90a13e518e4e-plugins-config-volume" pod "6ac6b3d6-6972-4721-af33-90a13e518e4e" (UID: "6ac6b3d6-6972-4721-af33-90a13e518e4e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:50.841218 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.841183    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf" event={"ID":"6ac6b3d6-6972-4721-af33-90a13e518e4e","Type":"ContainerDied","Data":"de30f07373ccc4b1261e972fb1da3d2a892144840d28a8cd9cdebeb79ba40b0f"}
Oct 30 18:58:50.841326 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.841248    4925 scope.go:122] "RemoveContainer" containerID="98cec756b152b16181640fe95b3aad1e7d0d276971579e50b58d7408dfd0c452"
Oct 30 18:58:50.841397 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.841366    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"
Oct 30 18:58:50.855752 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.855429    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:58:50.861324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.861303    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"
Oct 30 18:58:50.881814 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.881761    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z" podStartSLOduration=5.881741845 podStartE2EDuration="5.881741845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:50.854359658 +0000 UTC m=+1352.570704747" watchObservedRunningTime="2026-10-30 18:58:50.881741845 +0000 UTC m=+1352.598086923"
Oct 30 18:58:50.915553 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.914981    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q" podStartSLOduration=5.914965243 podStartE2EDuration="5.914965243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:50.911011206 +0000 UTC m=+1352.627356282" watchObservedRunningTime="2026-10-30 18:58:50.914965243 +0000 UTC m=+1352.631310320"
Oct 30 18:58:50.918643 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.917889    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ac6b3d6-6972-4721-af33-90a13e518e4e-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:58:50.979122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.978830    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"]
Oct 30 18:58:50.998355 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:50.998316    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-5ffbfd647-2k8mf"]
Oct 30 18:58:51.003970 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:51.003945    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"]
Oct 30 18:58:51.018468 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:51.018435    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe0612-55bdc47886-fbksg"]
Oct 30 18:58:51.421567 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:51.421490    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:51.878173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:51.877398    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q" event={"ID":"89911b2c-ec95-4aa1-b6cc-5c712d5cabae","Type":"ContainerStarted","Data":"9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154"}
Oct 30 18:58:52.095500 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.095418    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:52.428207 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.428168    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ac6b3d6-6972-4721-af33-90a13e518e4e" path="/var/lib/kubelet/pods/6ac6b3d6-6972-4721-af33-90a13e518e4e/volumes"
Oct 30 18:58:52.428853 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.428833    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73b71a46-e413-45f2-a484-415344e068d3" path="/var/lib/kubelet/pods/73b71a46-e413-45f2-a484-415344e068d3/volumes"
Oct 30 18:58:52.753899 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.753791    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:52.838505 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.838469    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"]
Oct 30 18:58:52.839435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.839414    4925 cpu_manager.go:414] "containerMap: removing container" podUID="73b71a46-e413-45f2-a484-415344e068d3" containerName="payload-processing"
Oct 30 18:58:52.839435 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.839435    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73b71a46-e413-45f2-a484-415344e068d3" containerName="payload-processing"
Oct 30 18:58:52.839652 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.839461    4925 cpu_manager.go:414] "containerMap: removing container" podUID="6ac6b3d6-6972-4721-af33-90a13e518e4e" containerName="payload-pre-processing"
Oct 30 18:58:52.839652 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.839470    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ac6b3d6-6972-4721-af33-90a13e518e4e" containerName="payload-pre-processing"
Oct 30 18:58:52.839753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.839726    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="73b71a46-e413-45f2-a484-415344e068d3" containerName="payload-processing"
Oct 30 18:58:52.839753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.839744    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ac6b3d6-6972-4721-af33-90a13e518e4e" containerName="payload-pre-processing"
Oct 30 18:58:52.844990 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.844965    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"
Oct 30 18:58:52.855173 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.855145    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"]
Oct 30 18:58:52.966313 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.964177    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw2zh\" (UniqueName: \"kubernetes.io/projected/da3032fe-24ff-41ec-8798-93a6ac6a0420-kube-api-access-rw2zh\") pod \"payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx\" (UID: \"da3032fe-24ff-41ec-8798-93a6ac6a0420\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"
Oct 30 18:58:52.966313 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:52.964303    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3032fe-24ff-41ec-8798-93a6ac6a0420-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx\" (UID: \"da3032fe-24ff-41ec-8798-93a6ac6a0420\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"
Oct 30 18:58:53.011058 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.010927    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:53.018298 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.012867    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-fe0612-dockercfg-ft48c\""
Oct 30 18:58:53.099178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.018675    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-fe0612\""
Oct 30 18:58:53.099178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.036704    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"]
Oct 30 18:58:53.099178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.067448    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw2zh\" (UniqueName: \"kubernetes.io/projected/da3032fe-24ff-41ec-8798-93a6ac6a0420-kube-api-access-rw2zh\") pod \"payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx\" (UID: \"da3032fe-24ff-41ec-8798-93a6ac6a0420\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"
Oct 30 18:58:53.099178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.093404    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3032fe-24ff-41ec-8798-93a6ac6a0420-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx\" (UID: \"da3032fe-24ff-41ec-8798-93a6ac6a0420\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"
Oct 30 18:58:53.099178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.094480    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3032fe-24ff-41ec-8798-93a6ac6a0420-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx\" (UID: \"da3032fe-24ff-41ec-8798-93a6ac6a0420\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"
Oct 30 18:58:53.121924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.105307    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"]
Oct 30 18:58:53.121924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.107428    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"
Oct 30 18:58:53.121924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.121846    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw2zh\" (UniqueName: \"kubernetes.io/projected/da3032fe-24ff-41ec-8798-93a6ac6a0420-kube-api-access-rw2zh\") pod \"payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx\" (UID: \"da3032fe-24ff-41ec-8798-93a6ac6a0420\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"
Oct 30 18:58:53.196487 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.195809    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg45p\" (UniqueName: \"kubernetes.io/projected/6153848e-01ea-40f0-ad84-70ebd26a4d32-kube-api-access-pg45p\") pod \"payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc\" (UID: \"6153848e-01ea-40f0-ad84-70ebd26a4d32\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"
Oct 30 18:58:53.196487 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.195897    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6153848e-01ea-40f0-ad84-70ebd26a4d32-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc\" (UID: \"6153848e-01ea-40f0-ad84-70ebd26a4d32\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"
Oct 30 18:58:53.297171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.297051    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg45p\" (UniqueName: \"kubernetes.io/projected/6153848e-01ea-40f0-ad84-70ebd26a4d32-kube-api-access-pg45p\") pod \"payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc\" (UID: \"6153848e-01ea-40f0-ad84-70ebd26a4d32\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"
Oct 30 18:58:53.297171 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.297151    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6153848e-01ea-40f0-ad84-70ebd26a4d32-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc\" (UID: \"6153848e-01ea-40f0-ad84-70ebd26a4d32\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"
Oct 30 18:58:53.299327 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.299296    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6153848e-01ea-40f0-ad84-70ebd26a4d32-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc\" (UID: \"6153848e-01ea-40f0-ad84-70ebd26a4d32\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"
Oct 30 18:58:53.316088 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.315965    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg45p\" (UniqueName: \"kubernetes.io/projected/6153848e-01ea-40f0-ad84-70ebd26a4d32-kube-api-access-pg45p\") pod \"payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc\" (UID: \"6153848e-01ea-40f0-ad84-70ebd26a4d32\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"
Oct 30 18:58:53.383135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.377410    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"
Oct 30 18:58:53.383135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.381031    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:53.455926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.455710    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"
Oct 30 18:58:53.723853 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:58:53.723820    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda3032fe_24ff_41ec_8798_93a6ac6a0420.slice/crio-44a39116a39850ca26380d6c328386ce510ecee3b08cac3201b4a1fef675e3c9 WatchSource:0}: Error finding container 44a39116a39850ca26380d6c328386ce510ecee3b08cac3201b4a1fef675e3c9: Status 404 returned error can't find the container with id 44a39116a39850ca26380d6c328386ce510ecee3b08cac3201b4a1fef675e3c9
Oct 30 18:58:53.724577 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.724551    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"]
Oct 30 18:58:53.811885 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:58:53.811748    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6153848e_01ea_40f0_ad84_70ebd26a4d32.slice/crio-0e72ecdc2577fc635c0db60279b195a042b525eefa7709acf91160fe60f6c378 WatchSource:0}: Error finding container 0e72ecdc2577fc635c0db60279b195a042b525eefa7709acf91160fe60f6c378: Status 404 returned error can't find the container with id 0e72ecdc2577fc635c0db60279b195a042b525eefa7709acf91160fe60f6c378
Oct 30 18:58:53.815676 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:53.814953    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"]
Oct 30 18:58:54.141162 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:54.141066    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx" event={"ID":"da3032fe-24ff-41ec-8798-93a6ac6a0420","Type":"ContainerStarted","Data":"1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a"}
Oct 30 18:58:54.141162 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:54.141145    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx" event={"ID":"da3032fe-24ff-41ec-8798-93a6ac6a0420","Type":"ContainerStarted","Data":"44a39116a39850ca26380d6c328386ce510ecee3b08cac3201b4a1fef675e3c9"}
Oct 30 18:58:54.141672 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:54.141394    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"
Oct 30 18:58:54.144007 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:54.143973    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc" event={"ID":"6153848e-01ea-40f0-ad84-70ebd26a4d32","Type":"ContainerStarted","Data":"a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc"}
Oct 30 18:58:54.144166 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:54.144029    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc" event={"ID":"6153848e-01ea-40f0-ad84-70ebd26a4d32","Type":"ContainerStarted","Data":"0e72ecdc2577fc635c0db60279b195a042b525eefa7709acf91160fe60f6c378"}
Oct 30 18:58:54.144720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:54.144697    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"
Oct 30 18:58:54.187926 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:54.186774    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx" podStartSLOduration=2.186754491 podStartE2EDuration="2.186754491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:54.159387536 +0000 UTC m=+1355.875732616" watchObservedRunningTime="2026-10-30 18:58:54.186754491 +0000 UTC m=+1355.903099569"
Oct 30 18:58:54.192194 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:54.190279    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc" podStartSLOduration=2.190261871 podStartE2EDuration="2.190261871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:54.181661615 +0000 UTC m=+1355.898006691" watchObservedRunningTime="2026-10-30 18:58:54.190261871 +0000 UTC m=+1355.906606949"
Oct 30 18:58:54.252284 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:54.252239    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:54.689084 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:54.689023    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:55.131560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:55.131281    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:55.392243 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:55.391901    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:55.700951 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:55.700691    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:56.153388 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:56.153236    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:58:56.167317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:56.155153    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:58:56.290402 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:56.290367    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:57.129808 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:57.128310    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:57.794380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:57.701327    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": read tcp 10.132.0.2:41618->10.132.0.136:15021: read: connection reset by peer" start-of-body=
Oct 30 18:58:57.794380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:57.701420    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": read tcp 10.132.0.2:41618->10.132.0.136:15021: read: connection reset by peer"
Oct 30 18:58:57.794380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:57.773764    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:58.316923 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.316891    4925 generic.go:356] "Generic (PLEG): container finished" podID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerID="59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d" exitCode=0
Oct 30 18:58:58.317473 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.316973    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" event={"ID":"0a67c842-7520-4af1-86b3-ef20d5bb6149","Type":"ContainerDied","Data":"59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d"}
Oct 30 18:58:58.317473 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.317455    4925 scope.go:122] "RemoveContainer" containerID="59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d"
Oct 30 18:58:58.319221 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.319198    4925 generic.go:356] "Generic (PLEG): container finished" podID="415056de-18a9-46fc-97cd-db55c15ce521" containerID="9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7" exitCode=0
Oct 30 18:58:58.319368 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.319268    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" event={"ID":"415056de-18a9-46fc-97cd-db55c15ce521","Type":"ContainerDied","Data":"9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7"}
Oct 30 18:58:58.319451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.319388    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:58.319512 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.319484    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:58.319572 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.319540    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:58.319737 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.319712    4925 scope.go:122] "RemoveContainer" containerID="9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7"
Oct 30 18:58:58.322141 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.322116    4925 generic.go:356] "Generic (PLEG): container finished" podID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerID="07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858" exitCode=0
Oct 30 18:58:58.322269 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.322194    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" event={"ID":"d699d780-921f-48ae-8ce4-27fdd806b52d","Type":"ContainerDied","Data":"07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858"}
Oct 30 18:58:58.322269 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.322246    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:58.322417 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.322325    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:58.322417 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.322378    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:58.322591 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.322562    4925 scope.go:122] "RemoveContainer" containerID="07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858"
Oct 30 18:58:58.324439 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.324397    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:58.324560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.324480    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:58.324560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.324530    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:58:58.801369 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:58.801279    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:59.394127 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:59.393986    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:59.417007 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:59.416962    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" event={"ID":"415056de-18a9-46fc-97cd-db55c15ce521","Type":"ContainerStarted","Data":"fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a"}
Oct 30 18:58:59.473131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:59.469483    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" event={"ID":"d699d780-921f-48ae-8ce4-27fdd806b52d","Type":"ContainerStarted","Data":"6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9"}
Oct 30 18:58:59.664184 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:59.656085    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:59.705122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:59.693650    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" event={"ID":"0a67c842-7520-4af1-86b3-ef20d5bb6149","Type":"ContainerStarted","Data":"bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002"}
Oct 30 18:58:59.734546 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:59.734512    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:58:59.734879 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:59.734857    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Oct 30 18:58:59.735078 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:58:59.735053    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Oct 30 18:59:00.160622 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.160590    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:59:00.167286 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.167169    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Oct 30 18:59:00.167286 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.167227    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Oct 30 18:59:00.514252 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.514157    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:59:00.521354 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.515448    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 18:59:00.521578 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.521554    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 18:59:00.609122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.605488    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"]
Oct 30 18:59:00.755329 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.755287    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Oct 30 18:59:00.755546 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.755352    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Oct 30 18:59:00.778783 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.778685    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"]
Oct 30 18:59:00.779152 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.779131    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:00.850415 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.850371    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-10d367-serving-cert\""
Oct 30 18:59:00.898809 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.898261    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-metrics-certs\") pod \"maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:00.898809 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.898456    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6cd5\" (UniqueName: \"kubernetes.io/projected/03a6f6b5-669d-449a-86b4-bbf8d0329f58-kube-api-access-p6cd5\") pod \"maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:00.898809 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.898563    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a6f6b5-669d-449a-86b4-bbf8d0329f58-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:00.898809 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.898628    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-maas-api-tls\") pod \"maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:00.942603 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:00.942481    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:01.025718 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.025682    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6cd5\" (UniqueName: \"kubernetes.io/projected/03a6f6b5-669d-449a-86b4-bbf8d0329f58-kube-api-access-p6cd5\") pod \"maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:01.026304 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.026278    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a6f6b5-669d-449a-86b4-bbf8d0329f58-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:01.026730 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.026527    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-maas-api-tls\") pod \"maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:01.052417 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.052332    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-metrics-certs\") pod \"maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:01.077793 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.077752    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a6f6b5-669d-449a-86b4-bbf8d0329f58-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:01.141133 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.113888    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-metrics-certs\") pod \"maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:01.166872 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.147649    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-maas-api-tls\") pod \"maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:01.172120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.168620    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6cd5\" (UniqueName: \"kubernetes.io/projected/03a6f6b5-669d-449a-86b4-bbf8d0329f58-kube-api-access-p6cd5\") pod \"maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:01.197147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.173227    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Oct 30 18:59:01.197147 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.173278    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Oct 30 18:59:01.224144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.210958    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:01.444124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.443691    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:01.517149 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.516638    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 18:59:01.517149 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.516714    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 18:59:01.657520 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.656271    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:59:01.763178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.758240    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Oct 30 18:59:01.763178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:01.758341    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Oct 30 18:59:02.296183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:02.182375    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:02.296183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:02.184752    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Oct 30 18:59:02.296183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:02.184808    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Oct 30 18:59:02.802427 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:02.737269    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:02.853240 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:02.830596    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:02.871135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:02.862193    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:02.947693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:02.929887    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:02.947693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:02.930482    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:02.947693 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:02.946194    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm" podUID="47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.128:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:03.126965 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.124877    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:03.130036 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.129995    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Oct 30 18:59:03.295172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.292047    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Oct 30 18:59:03.340721 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.323193    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:03.403180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.387597    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Oct 30 18:59:03.403180 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.387655    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Oct 30 18:59:03.405424 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.405391    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"]
Oct 30 18:59:03.430793 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.203414    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/echo-simulator-kserve-64b6f8f96c-qht75" podUID="bd42a605-1c88-4212-9199-1f3572a08984" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:03.431357 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.203485    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 18:59:03.431586 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.431544    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 18:59:03.532009 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.531973    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 18:59:03.586297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.586255    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 18:59:03.752121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.739187    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:03.824224 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.824186    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"
Oct 30 18:59:03.824905 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.824888    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"
Oct 30 18:59:03.825087 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.825063    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:59:03.825222 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.825210    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"]
Oct 30 18:59:03.833859 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.833829    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"
Oct 30 18:59:03.861829 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.861786    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Oct 30 18:59:03.862022 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.861867    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Oct 30 18:59:03.877526 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.877483    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:03.878586 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.878563    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:59:03.902807 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.902768    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-3d1aa8\""
Oct 30 18:59:03.903015 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.902932    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-3d1aa8-dockercfg-qfkcj\""
Oct 30 18:59:03.913297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.913265    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"]
Oct 30 18:59:03.948610 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.948574    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:04.009270 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.950056    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lggs\" (UniqueName: \"kubernetes.io/projected/8184da4d-3c5f-4628-af7d-f5e70b2029ad-kube-api-access-7lggs\") pod \"payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv\" (UID: \"8184da4d-3c5f-4628-af7d-f5e70b2029ad\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"
Oct 30 18:59:04.177872 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.177834    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8184da4d-3c5f-4628-af7d-f5e70b2029ad-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv\" (UID: \"8184da4d-3c5f-4628-af7d-f5e70b2029ad\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"
Oct 30 18:59:04.178795 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:03.991931    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 18:59:04.203317 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.161320    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Oct 30 18:59:04.369644 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.369195    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Oct 30 18:59:04.369644 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.283012    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8184da4d-3c5f-4628-af7d-f5e70b2029ad-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv\" (UID: \"8184da4d-3c5f-4628-af7d-f5e70b2029ad\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"
Oct 30 18:59:04.369644 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.369506    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lggs\" (UniqueName: \"kubernetes.io/projected/8184da4d-3c5f-4628-af7d-f5e70b2029ad-kube-api-access-7lggs\") pod \"payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv\" (UID: \"8184da4d-3c5f-4628-af7d-f5e70b2029ad\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"
Oct 30 18:59:04.394121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.267029    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 18:59:04.432977 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.432930    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:04.443257 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.437947    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:59:04.447119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.443391    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"
Oct 30 18:59:04.450044 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.283894    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8184da4d-3c5f-4628-af7d-f5e70b2029ad-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv\" (UID: \"8184da4d-3c5f-4628-af7d-f5e70b2029ad\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"
Oct 30 18:59:04.537129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.532758    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfm5r\" (UniqueName: \"kubernetes.io/projected/450d1376-252e-4a85-8b48-3337bce4e5d3-kube-api-access-vfm5r\") pod \"payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs\" (UID: \"450d1376-252e-4a85-8b48-3337bce4e5d3\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"
Oct 30 18:59:04.537129 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.532862    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/450d1376-252e-4a85-8b48-3337bce4e5d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs\" (UID: \"450d1376-252e-4a85-8b48-3337bce4e5d3\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"
Oct 30 18:59:04.580808 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.577797    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:04.771788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.700304    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 18:59:04.771788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.700376    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 18:59:04.771788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.710321    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"]
Oct 30 18:59:04.771788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.710397    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"]
Oct 30 18:59:04.771788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.710468    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 18:59:04.771788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.710525    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:59:04.771788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.710559    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:59:04.771788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.710575    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks" event={"ID":"03a6f6b5-669d-449a-86b4-bbf8d0329f58","Type":"ContainerStarted","Data":"f62b8d8e058c0e445fca499453cdb0dcb3d6d42fb9cad3ca1d6ada4525406e7b"}
Oct 30 18:59:04.771788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.760519    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Oct 30 18:59:04.771788 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.760590    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Oct 30 18:59:04.843402 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.810336    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 18:59:04.880123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.863775    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lggs\" (UniqueName: \"kubernetes.io/projected/8184da4d-3c5f-4628-af7d-f5e70b2029ad-kube-api-access-7lggs\") pod \"payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv\" (UID: \"8184da4d-3c5f-4628-af7d-f5e70b2029ad\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"
Oct 30 18:59:04.880463 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.880422    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 18:59:04.910210 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.910048    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"
Oct 30 18:59:04.966900 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.966858    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfm5r\" (UniqueName: \"kubernetes.io/projected/450d1376-252e-4a85-8b48-3337bce4e5d3-kube-api-access-vfm5r\") pod \"payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs\" (UID: \"450d1376-252e-4a85-8b48-3337bce4e5d3\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"
Oct 30 18:59:04.971137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.967021    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/450d1376-252e-4a85-8b48-3337bce4e5d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs\" (UID: \"450d1376-252e-4a85-8b48-3337bce4e5d3\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"
Oct 30 18:59:04.996522 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:04.976579    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/450d1376-252e-4a85-8b48-3337bce4e5d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs\" (UID: \"450d1376-252e-4a85-8b48-3337bce4e5d3\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"
Oct 30 18:59:05.172678 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.172646    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 18:59:05.199265 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.199231    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfm5r\" (UniqueName: \"kubernetes.io/projected/450d1376-252e-4a85-8b48-3337bce4e5d3-kube-api-access-vfm5r\") pod \"payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs\" (UID: \"450d1376-252e-4a85-8b48-3337bce4e5d3\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"
Oct 30 18:59:05.218504 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.218468    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Oct 30 18:59:05.218740 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.218712    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Oct 30 18:59:05.349119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.345312    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"
Oct 30 18:59:05.395122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.370327    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:05.425395 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.425313    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"
Oct 30 18:59:05.437121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.430427    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:05.437121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.430736    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 18:59:05.474124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.449001    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:05.484666 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.484620    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 18:59:05.598400 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.598364    4925 patch_prober.go:29] interesting pod/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 18:59:05.638385 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.638347    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 18:59:05.797529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.760931    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Oct 30 18:59:05.797529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.760983    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Oct 30 18:59:06.003177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:05.971196    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:59:06.037176 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.037087    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 18:59:06.318224 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.315682    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"
Oct 30 18:59:06.373765 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.373729    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:59:06.385471 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.385400    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 18:59:06.526478 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.526436    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:59:06.527630 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.527605    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:59:06.528840 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.528788    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:59:06.538446 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.538423    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"]
Oct 30 18:59:06.560953 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.560882    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"]
Oct 30 18:59:06.726533 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.726504    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:59:06.727122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.727079    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:59:06.727855 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.727834    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:59:06.806250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.805930    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv" event={"ID":"8184da4d-3c5f-4628-af7d-f5e70b2029ad","Type":"ContainerStarted","Data":"521378f8af3487a8708812f13ddada50b12ffd9850880a10c89398e551c6842f"}
Oct 30 18:59:06.806250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.806069    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv" event={"ID":"8184da4d-3c5f-4628-af7d-f5e70b2029ad","Type":"ContainerStarted","Data":"ed3259f3b15ec7490087100e6752d0b9ad03ad6f7f64e8d0889c34096f2853c4"}
Oct 30 18:59:06.806250 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.806195    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"
Oct 30 18:59:06.808823 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.808796    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks" event={"ID":"03a6f6b5-669d-449a-86b4-bbf8d0329f58","Type":"ContainerStarted","Data":"af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20"}
Oct 30 18:59:06.809270 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.809252    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:06.812416 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.812394    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs" event={"ID":"450d1376-252e-4a85-8b48-3337bce4e5d3","Type":"ContainerStarted","Data":"fe8d0181b99a61229d0b05c789ee48d297feddcb6ec8f3f7fdddd3568ef64cde"}
Oct 30 18:59:06.812529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.812439    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"
Oct 30 18:59:06.812529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.812457    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs" event={"ID":"450d1376-252e-4a85-8b48-3337bce4e5d3","Type":"ContainerStarted","Data":"f465706ef3d7803f87b7e51e3d47c8a3172904f06193e545b2aa4042065f75a7"}
Oct 30 18:59:06.827047 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.826965    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv" podStartSLOduration=4.826953497 podStartE2EDuration="4.826953497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:06.822999568 +0000 UTC m=+1368.539344650" watchObservedRunningTime="2026-10-30 18:59:06.826953497 +0000 UTC m=+1368.543298609"
Oct 30 18:59:06.846047 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.846000    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks" podStartSLOduration=5.622163942 podStartE2EDuration="6.845985293s" totalImagesPullingTime="1.223821351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:06.84550213 +0000 UTC m=+1368.561847204" watchObservedRunningTime="2026-10-30 18:59:06.845985293 +0000 UTC m=+1368.562330370"
Oct 30 18:59:06.850559 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.850525    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:06.866277 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:06.866238    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs" podStartSLOduration=4.866226432 podStartE2EDuration="4.866226432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:06.862499639 +0000 UTC m=+1368.578844705" watchObservedRunningTime="2026-10-30 18:59:06.866226432 +0000 UTC m=+1368.582571508"
Oct 30 18:59:07.704201 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:07.704160    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:08.474469 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:08.474436    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:09.073725 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:09.073689    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:09.412500 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:09.412463    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:10.044797 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:10.042990    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:10.873139 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:10.872864    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:11.461153 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:11.456201    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:12.120130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:12.118399    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:12.824020 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:12.823994    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 18:59:12.938766 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:12.938731    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:13.311339 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:13.311302    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:59:13.693428 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:13.688284    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:14.248644 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.248610    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"]
Oct 30 18:59:14.254911 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.254887    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.266592 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.266566    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-3d1aa8-serving-cert\""
Oct 30 18:59:14.268458 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.268432    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"]
Oct 30 18:59:14.365528 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.365494    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:14.419464 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.419419    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-metrics-certs\") pod \"maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.473121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.468178    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvjfr\" (UniqueName: \"kubernetes.io/projected/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-kube-api-access-jvjfr\") pod \"maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.473121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.468253    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-maas-api-tls\") pod \"maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.473121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.468351    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.570172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.569928    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-metrics-certs\") pod \"maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.570172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.569998    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvjfr\" (UniqueName: \"kubernetes.io/projected/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-kube-api-access-jvjfr\") pod \"maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.570172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.570031    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:14.570172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.570055    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-maas-api-tls\") pod \"maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.570172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.570143    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.572773 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.571084    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.574850 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.574801    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-maas-api-tls\") pod \"maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.578159 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.577972    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-metrics-certs\") pod \"maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.598602 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.598536    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvjfr\" (UniqueName: \"kubernetes.io/projected/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-kube-api-access-jvjfr\") pod \"maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:14.879763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:14.878387    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:15.325319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:15.325287    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"]
Oct 30 18:59:15.326193 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:59:15.326170    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bcc374b_bcc5_4ddd_ae99_22b8b53ae078.slice/crio-4c35d6b3514fe3748592f686e3d3113577be96c100224c026879b2b139b7d07a WatchSource:0}: Error finding container 4c35d6b3514fe3748592f686e3d3113577be96c100224c026879b2b139b7d07a: Status 404 returned error can't find the container with id 4c35d6b3514fe3748592f686e3d3113577be96c100224c026879b2b139b7d07a
Oct 30 18:59:15.400320 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:15.400289    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:15.983159 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:15.980901    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp" event={"ID":"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078","Type":"ContainerStarted","Data":"d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c"}
Oct 30 18:59:15.983159 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:15.980973    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp" event={"ID":"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078","Type":"ContainerStarted","Data":"4c35d6b3514fe3748592f686e3d3113577be96c100224c026879b2b139b7d07a"}
Oct 30 18:59:15.983159 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:15.981034    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:16.021465 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:16.021394    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp" podStartSLOduration=1.518009379 podStartE2EDuration="2.021375583s" totalImagesPullingTime="503.366204ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:16.018749843 +0000 UTC m=+1377.735094921" watchObservedRunningTime="2026-10-30 18:59:16.021375583 +0000 UTC m=+1377.737720662"
Oct 30 18:59:16.509126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:16.506675    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:17.520119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:17.516380    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:17.821662 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:17.821584    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"
Oct 30 18:59:17.821835 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:17.821666    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"
Oct 30 18:59:18.589120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:18.587359    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:19.443366 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:19.443324    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:19.626240 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:19.626208    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:20.017862 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:20.017769    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"]
Oct 30 18:59:20.018048 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:20.018013    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr" podUID="0bef6ca6-3659-49e8-8fa7-b3984d0be9a4" containerName="payload-processing" containerID="cri-o://ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a" gracePeriod=30
Oct 30 18:59:20.025478 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:20.025441    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"]
Oct 30 18:59:20.025672 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:20.025646    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5" podUID="0f6a689f-9e21-473c-b918-e2a05e18c87f" containerName="payload-pre-processing" containerID="cri-o://566be32b80ab735f73bda0a5642f8f85c99d677c13afb6d0898c8911c666b199" gracePeriod=30
Oct 30 18:59:20.117126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:20.113341    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:20.315081 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:20.314946    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:20.535242 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:20.535210    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:20.945220 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:20.945081    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:21.020017 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.019992    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"
Oct 30 18:59:21.047820 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.047783    4925 generic.go:356] "Generic (PLEG): container finished" podID="0bef6ca6-3659-49e8-8fa7-b3984d0be9a4" containerID="ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a" exitCode=0
Oct 30 18:59:21.047982 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.047862    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr" event={"ID":"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4","Type":"ContainerDied","Data":"ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a"}
Oct 30 18:59:21.048123 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.048106    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"
Oct 30 18:59:21.048543 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.048518    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr" event={"ID":"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4","Type":"ContainerDied","Data":"88d409184d372b77cafec0efee5a2d6a2df4d669d648eb203dfaa815403d601e"}
Oct 30 18:59:21.048661 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.048574    4925 scope.go:122] "RemoveContainer" containerID="ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a"
Oct 30 18:59:21.051468 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.051343    4925 generic.go:356] "Generic (PLEG): container finished" podID="0f6a689f-9e21-473c-b918-e2a05e18c87f" containerID="566be32b80ab735f73bda0a5642f8f85c99d677c13afb6d0898c8911c666b199" exitCode=0
Oct 30 18:59:21.051576 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.051526    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5" event={"ID":"0f6a689f-9e21-473c-b918-e2a05e18c87f","Type":"ContainerDied","Data":"566be32b80ab735f73bda0a5642f8f85c99d677c13afb6d0898c8911c666b199"}
Oct 30 18:59:21.107337 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.107306    4925 scope.go:122] "RemoveContainer" containerID="ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a"
Oct 30 18:59:21.107800 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:21.107755    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a\": container with ID starting with ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a not found: ID does not exist" containerID="ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a"
Oct 30 18:59:21.107917 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.107806    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a"} err="failed to get container status \"ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a\": rpc error: code = NotFound desc = could not find container \"ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a\": container with ID starting with ecf82ccfe11b5498b4aafcf3e714b17f31a3cbadbc8726546304ed1147f4a17a not found: ID does not exist"
Oct 30 18:59:21.109811 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.109775    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-plugins-config-volume\") pod \"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4\" (UID: \"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4\") "
Oct 30 18:59:21.109953 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.109899    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-kube-api-access-wvdhl\" (UniqueName: \"kubernetes.io/projected/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-kube-api-access-wvdhl\") pod \"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4\" (UID: \"0bef6ca6-3659-49e8-8fa7-b3984d0be9a4\") "
Oct 30 18:59:21.110639 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.110614    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-plugins-config-volume" pod "0bef6ca6-3659-49e8-8fa7-b3984d0be9a4" (UID: "0bef6ca6-3659-49e8-8fa7-b3984d0be9a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:21.111928 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.111905    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:21.116292 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.116264    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-kube-api-access-wvdhl" pod "0bef6ca6-3659-49e8-8fa7-b3984d0be9a4" (UID: "0bef6ca6-3659-49e8-8fa7-b3984d0be9a4"). InnerVolumeSpecName "kube-api-access-wvdhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:21.129741 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.129717    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:21.211088 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.211017    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"
Oct 30 18:59:21.213434 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.213405    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvdhl\" (UniqueName: \"kubernetes.io/projected/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4-kube-api-access-wvdhl\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:21.288186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.288052    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-vfsrt_77704542-8e8e-4d3e-a4bd-e07d4f0e79a0/payload-processing/0.log"
Oct 30 18:59:21.314617 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.314586    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f6a689f-9e21-473c-b918-e2a05e18c87f-kube-api-access-blk2l\" (UniqueName: \"kubernetes.io/projected/0f6a689f-9e21-473c-b918-e2a05e18c87f-kube-api-access-blk2l\") pod \"0f6a689f-9e21-473c-b918-e2a05e18c87f\" (UID: \"0f6a689f-9e21-473c-b918-e2a05e18c87f\") "
Oct 30 18:59:21.314790 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.314768    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f6a689f-9e21-473c-b918-e2a05e18c87f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f6a689f-9e21-473c-b918-e2a05e18c87f-plugins-config-volume\") pod \"0f6a689f-9e21-473c-b918-e2a05e18c87f\" (UID: \"0f6a689f-9e21-473c-b918-e2a05e18c87f\") "
Oct 30 18:59:21.315731 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.315703    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f6a689f-9e21-473c-b918-e2a05e18c87f-plugins-config-volume" pod "0f6a689f-9e21-473c-b918-e2a05e18c87f" (UID: "0f6a689f-9e21-473c-b918-e2a05e18c87f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:21.317882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.317856    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f6a689f-9e21-473c-b918-e2a05e18c87f-kube-api-access-blk2l" pod "0f6a689f-9e21-473c-b918-e2a05e18c87f" (UID: "0f6a689f-9e21-473c-b918-e2a05e18c87f"). InnerVolumeSpecName "kube-api-access-blk2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:21.398324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.398141    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"]
Oct 30 18:59:21.400480 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.400452    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-54bc2b-78c54d9486-m8wgr"]
Oct 30 18:59:21.416343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.416321    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blk2l\" (UniqueName: \"kubernetes.io/projected/0f6a689f-9e21-473c-b918-e2a05e18c87f-kube-api-access-blk2l\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:21.416343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.416343    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f6a689f-9e21-473c-b918-e2a05e18c87f-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:21.507016 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.506943    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-vfsrt_77704542-8e8e-4d3e-a4bd-e07d4f0e79a0/payload-processing/0.log"
Oct 30 18:59:21.702569 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.702541    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6_c7eea034-caf9-4b41-8cbb-77715993d9d1/payload-processing/0.log"
Oct 30 18:59:21.820169 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.819164    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:21.903346 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.903315    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6_c7eea034-caf9-4b41-8cbb-77715993d9d1/payload-processing/0.log"
Oct 30 18:59:21.997472 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:21.997440    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 18:59:22.066205 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:22.066164    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5" event={"ID":"0f6a689f-9e21-473c-b918-e2a05e18c87f","Type":"ContainerDied","Data":"51c038ff09d9adc393e2957787217df0e52a24162e5b65af23832ecf521ff146"}
Oct 30 18:59:22.066400 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:22.066238    4925 scope.go:122] "RemoveContainer" containerID="566be32b80ab735f73bda0a5642f8f85c99d677c13afb6d0898c8911c666b199"
Oct 30 18:59:22.066400 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:22.066355    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"
Oct 30 18:59:22.096647 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:22.096622    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"]
Oct 30 18:59:22.102200 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:22.102175    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-54bc2b-75665fb467-9mcc5"]
Oct 30 18:59:22.425201 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:22.425172    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0bef6ca6-3659-49e8-8fa7-b3984d0be9a4" path="/var/lib/kubelet/pods/0bef6ca6-3659-49e8-8fa7-b3984d0be9a4/volumes"
Oct 30 18:59:22.425699 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:22.425683    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f6a689f-9e21-473c-b918-e2a05e18c87f" path="/var/lib/kubelet/pods/0f6a689f-9e21-473c-b918-e2a05e18c87f/volumes"
Oct 30 18:59:22.520120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:22.518271    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:23.670130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:23.667634    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:24.557126 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:24.556828    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:25.402178 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:25.399132    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:26.071121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:26.070736    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:26.111380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:26.111199    4925 generic.go:356] "Generic (PLEG): container finished" podID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerID="5cc8988293ddfd81bf7c75ae9876ae7ee4971ae29a78bf43966d459bb3e49f10" exitCode=0
Oct 30 18:59:26.111577 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:26.111481    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" event={"ID":"e41a16b6-cb27-412e-90a6-5374b69b8a76","Type":"ContainerDied","Data":"5cc8988293ddfd81bf7c75ae9876ae7ee4971ae29a78bf43966d459bb3e49f10"}
Oct 30 18:59:26.113440 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:26.111972    4925 scope.go:122] "RemoveContainer" containerID="5cc8988293ddfd81bf7c75ae9876ae7ee4971ae29a78bf43966d459bb3e49f10"
Oct 30 18:59:26.115732 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:26.115025    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:59:26.115732 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:26.115124    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:59:26.115732 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:26.115171    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:59:27.007948 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:27.000753    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:27.124455 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:27.124396    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" event={"ID":"e41a16b6-cb27-412e-90a6-5374b69b8a76","Type":"ContainerStarted","Data":"d51ecb472ffb43ca243ada97a27c657ee3979e0b984bd73e262cd63585f12d49"}
Oct 30 18:59:27.702900 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:27.698270    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:27.753121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:27.751958    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:59:27.753121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:27.752113    4925 patch_prober.go:29] interesting pod/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 30 18:59:27.753121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:27.752158    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 30 18:59:27.894125 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:27.893182    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:28.150305 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:28.150219    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:28.525803 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:28.525773    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:59:28.749858 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:28.749784    4925 patch_prober.go:29] interesting pod/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 30 18:59:28.749858 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:28.749833    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 30 18:59:28.927118 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:28.920683    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:29.751135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:29.750730    4925 patch_prober.go:29] interesting pod/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 30 18:59:29.751135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:29.750795    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 30 18:59:30.014186 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:30.008838    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:30.750486 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:30.750451    4925 patch_prober.go:29] interesting pod/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 30 18:59:30.750668 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:30.750516    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 30 18:59:31.062975 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:31.054819    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:31.754701 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:31.754652    4925 patch_prober.go:29] interesting pod/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused" start-of-body=
Oct 30 18:59:31.755625 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:31.755578    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.147:15021/healthz/ready\": dial tcp 10.132.0.147:15021: connect: connection refused"
Oct 30 18:59:31.785121 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:31.778250    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:32.109138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:32.109024    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:32.294812 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:32.294777    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:32.546124 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:32.539904    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:32.835052 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:32.834968    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:59:32.836388 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:32.836366    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 18:59:33.202119 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:33.192535    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:33.791841 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:33.791803    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"]
Oct 30 18:59:33.808161 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:33.808129    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"]
Oct 30 18:59:33.955841 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:33.955793    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:34.164551 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:34.157072    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:34.891280 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:34.891248    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:35.185234 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:35.185201    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:35.767251 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:35.766757    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:36.382967 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:36.382929    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:36.617552 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:36.617513    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:36.844319 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:36.844280    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:37.097502 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:37.097408    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:37.688770 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:37.688730    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:38.461735 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:38.461700    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:38.793164 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:38.793064    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh" podUID="fd35ca47-1812-4719-ad92-5993b466e8c2" containerName="payload-processing" containerID="cri-o://ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c" gracePeriod=25
Oct 30 18:59:38.810607 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:38.810546    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj" podUID="ad8ce530-37d0-48f6-805d-1b034492b4fe" containerName="payload-pre-processing" containerID="cri-o://d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192" gracePeriod=25
Oct 30 18:59:39.127170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.078683    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:39.230495 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.230473    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"
Oct 30 18:59:39.260183 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.260152    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"
Oct 30 18:59:39.262635 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.262599    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd35ca47-1812-4719-ad92-5993b466e8c2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd35ca47-1812-4719-ad92-5993b466e8c2-plugins-config-volume\") pod \"fd35ca47-1812-4719-ad92-5993b466e8c2\" (UID: \"fd35ca47-1812-4719-ad92-5993b466e8c2\") "
Oct 30 18:59:39.262766 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.262737    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd35ca47-1812-4719-ad92-5993b466e8c2-kube-api-access-xcvpc\" (UniqueName: \"kubernetes.io/projected/fd35ca47-1812-4719-ad92-5993b466e8c2-kube-api-access-xcvpc\") pod \"fd35ca47-1812-4719-ad92-5993b466e8c2\" (UID: \"fd35ca47-1812-4719-ad92-5993b466e8c2\") "
Oct 30 18:59:39.263452 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.263423    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd35ca47-1812-4719-ad92-5993b466e8c2-plugins-config-volume" pod "fd35ca47-1812-4719-ad92-5993b466e8c2" (UID: "fd35ca47-1812-4719-ad92-5993b466e8c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:39.265696 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.265667    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd35ca47-1812-4719-ad92-5993b466e8c2-kube-api-access-xcvpc" pod "fd35ca47-1812-4719-ad92-5993b466e8c2" (UID: "fd35ca47-1812-4719-ad92-5993b466e8c2"). InnerVolumeSpecName "kube-api-access-xcvpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:39.351519 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.351481    4925 generic.go:356] "Generic (PLEG): container finished" podID="fd35ca47-1812-4719-ad92-5993b466e8c2" containerID="ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c" exitCode=0
Oct 30 18:59:39.351706 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.351545    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"
Oct 30 18:59:39.351706 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.351574    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh" event={"ID":"fd35ca47-1812-4719-ad92-5993b466e8c2","Type":"ContainerDied","Data":"ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c"}
Oct 30 18:59:39.351706 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.351637    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh" event={"ID":"fd35ca47-1812-4719-ad92-5993b466e8c2","Type":"ContainerDied","Data":"5affec9414e02447731ad2d48207c9878142045fa8bbdd369bae2536b4ea9a32"}
Oct 30 18:59:39.351706 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.351667    4925 scope.go:122] "RemoveContainer" containerID="ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c"
Oct 30 18:59:39.353198 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.353163    4925 generic.go:356] "Generic (PLEG): container finished" podID="ad8ce530-37d0-48f6-805d-1b034492b4fe" containerID="d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192" exitCode=0
Oct 30 18:59:39.353324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.353211    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"
Oct 30 18:59:39.353324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.353267    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj" event={"ID":"ad8ce530-37d0-48f6-805d-1b034492b4fe","Type":"ContainerDied","Data":"d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192"}
Oct 30 18:59:39.353324 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.353308    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj" event={"ID":"ad8ce530-37d0-48f6-805d-1b034492b4fe","Type":"ContainerDied","Data":"c0c35ba8d2805a06108ba9270efae65f2a0ec474780c3d3c3b596c3565b4cfee"}
Oct 30 18:59:39.364033 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.363973    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad8ce530-37d0-48f6-805d-1b034492b4fe-kube-api-access-zdz4c\" (UniqueName: \"kubernetes.io/projected/ad8ce530-37d0-48f6-805d-1b034492b4fe-kube-api-access-zdz4c\") pod \"ad8ce530-37d0-48f6-805d-1b034492b4fe\" (UID: \"ad8ce530-37d0-48f6-805d-1b034492b4fe\") "
Oct 30 18:59:39.364128 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.364062    4925 scope.go:122] "RemoveContainer" containerID="ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c"
Oct 30 18:59:39.364192 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.364147    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad8ce530-37d0-48f6-805d-1b034492b4fe-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8ce530-37d0-48f6-805d-1b034492b4fe-plugins-config-volume\") pod \"ad8ce530-37d0-48f6-805d-1b034492b4fe\" (UID: \"ad8ce530-37d0-48f6-805d-1b034492b4fe\") "
Oct 30 18:59:39.364432 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:39.364409    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c\": container with ID starting with ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c not found: ID does not exist" containerID="ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c"
Oct 30 18:59:39.364516 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.364442    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c"} err="failed to get container status \"ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c\": rpc error: code = NotFound desc = could not find container \"ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c\": container with ID starting with ad067b3baaf305071054e20a6b0bb54b0d444d19e6e16ba792d0f5dc83af397c not found: ID does not exist"
Oct 30 18:59:39.364516 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.364463    4925 scope.go:122] "RemoveContainer" containerID="d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192"
Oct 30 18:59:39.364838 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.364816    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcvpc\" (UniqueName: \"kubernetes.io/projected/fd35ca47-1812-4719-ad92-5993b466e8c2-kube-api-access-xcvpc\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:39.364906 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.364846    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd35ca47-1812-4719-ad92-5993b466e8c2-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:39.365217 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.365192    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad8ce530-37d0-48f6-805d-1b034492b4fe-plugins-config-volume" pod "ad8ce530-37d0-48f6-805d-1b034492b4fe" (UID: "ad8ce530-37d0-48f6-805d-1b034492b4fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:39.366618 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.366596    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad8ce530-37d0-48f6-805d-1b034492b4fe-kube-api-access-zdz4c" pod "ad8ce530-37d0-48f6-805d-1b034492b4fe" (UID: "ad8ce530-37d0-48f6-805d-1b034492b4fe"). InnerVolumeSpecName "kube-api-access-zdz4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:39.376939 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.376920    4925 scope.go:122] "RemoveContainer" containerID="d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192"
Oct 30 18:59:39.377381 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:39.377325    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192\": container with ID starting with d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192 not found: ID does not exist" containerID="d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192"
Oct 30 18:59:39.377381 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.377352    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192"} err="failed to get container status \"d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192\": rpc error: code = NotFound desc = could not find container \"d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192\": container with ID starting with d69ecc94e4cfd14373b7cc646cf5f528791faaa3281f0756773745a1d9ac7192 not found: ID does not exist"
Oct 30 18:59:39.378446 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.378418    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"]
Oct 30 18:59:39.383923 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.383902    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-655193-d795fdd78-d5kjh"]
Oct 30 18:59:39.466775 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.466741    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdz4c\" (UniqueName: \"kubernetes.io/projected/ad8ce530-37d0-48f6-805d-1b034492b4fe-kube-api-access-zdz4c\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:39.466775 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.466777    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad8ce530-37d0-48f6-805d-1b034492b4fe-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:39.679499 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.679470    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"]
Oct 30 18:59:39.683054 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.683028    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-655193-6ddd86dcff-vwfxj"]
Oct 30 18:59:39.793629 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:39.793592    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:40.424670 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:40.424622    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad8ce530-37d0-48f6-805d-1b034492b4fe" path="/var/lib/kubelet/pods/ad8ce530-37d0-48f6-805d-1b034492b4fe/volumes"
Oct 30 18:59:40.425424 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:40.425402    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd35ca47-1812-4719-ad92-5993b466e8c2" path="/var/lib/kubelet/pods/fd35ca47-1812-4719-ad92-5993b466e8c2/volumes"
Oct 30 18:59:40.520209 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:40.519734    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:41.276155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:41.273484    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:41.833529 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:41.833478    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:42.061038 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:42.060984    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:42.753994 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:42.753962    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:43.290741 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:43.290703    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:43.309786 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:43.309739    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 18:59:43.563050 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:43.562970    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:43.579633 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:43.579601    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"]
Oct 30 18:59:43.579875 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:43.579851    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy" containerID="cri-o://fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a" gracePeriod=30
Oct 30 18:59:43.775226 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:43.765256    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:44.045733 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.045155    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"]
Oct 30 18:59:44.045733 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.045370    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f" podUID="afc1c28a-abd2-4def-a6d7-201732ee1eee" containerName="maas-api" containerID="cri-o://7d19164f7c7c6b3bcd5ce69664ac0775acf77bd4e64f5e8495905a236b49083f" gracePeriod=30
Oct 30 18:59:44.201882 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.201701    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:44.450122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.449863    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:44.543942 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.543788    4925 generic.go:356] "Generic (PLEG): container finished" podID="afc1c28a-abd2-4def-a6d7-201732ee1eee" containerID="7d19164f7c7c6b3bcd5ce69664ac0775acf77bd4e64f5e8495905a236b49083f" exitCode=0
Oct 30 18:59:44.563423 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.563385    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f" event={"ID":"afc1c28a-abd2-4def-a6d7-201732ee1eee","Type":"ContainerDied","Data":"7d19164f7c7c6b3bcd5ce69664ac0775acf77bd4e64f5e8495905a236b49083f"}
Oct 30 18:59:44.637821 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.637793    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:59:44.675906 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.675866    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afc1c28a-abd2-4def-a6d7-201732ee1eee-kube-api-access-ljq5b\" (UniqueName: \"kubernetes.io/projected/afc1c28a-abd2-4def-a6d7-201732ee1eee-kube-api-access-ljq5b\") pod \"afc1c28a-abd2-4def-a6d7-201732ee1eee\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") "
Oct 30 18:59:44.676139 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.676052    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-maas-api-tls\") pod \"afc1c28a-abd2-4def-a6d7-201732ee1eee\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") "
Oct 30 18:59:44.676139 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.676112    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/afc1c28a-abd2-4def-a6d7-201732ee1eee-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc1c28a-abd2-4def-a6d7-201732ee1eee-openshift-service-ca\") pod \"afc1c28a-abd2-4def-a6d7-201732ee1eee\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") "
Oct 30 18:59:44.676308 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.676144    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-metrics-certs\") pod \"afc1c28a-abd2-4def-a6d7-201732ee1eee\" (UID: \"afc1c28a-abd2-4def-a6d7-201732ee1eee\") "
Oct 30 18:59:44.676924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.676898    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afc1c28a-abd2-4def-a6d7-201732ee1eee-openshift-service-ca" pod "afc1c28a-abd2-4def-a6d7-201732ee1eee" (UID: "afc1c28a-abd2-4def-a6d7-201732ee1eee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:44.678364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.678337    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc1c28a-abd2-4def-a6d7-201732ee1eee-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:44.680480 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.680451    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afc1c28a-abd2-4def-a6d7-201732ee1eee-kube-api-access-ljq5b" pod "afc1c28a-abd2-4def-a6d7-201732ee1eee" (UID: "afc1c28a-abd2-4def-a6d7-201732ee1eee"). InnerVolumeSpecName "kube-api-access-ljq5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:44.684258 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.684232    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-metrics-certs" pod "afc1c28a-abd2-4def-a6d7-201732ee1eee" (UID: "afc1c28a-abd2-4def-a6d7-201732ee1eee"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:59:44.684381 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.684297    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-maas-api-tls" pod "afc1c28a-abd2-4def-a6d7-201732ee1eee" (UID: "afc1c28a-abd2-4def-a6d7-201732ee1eee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:59:44.779794 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.779712    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljq5b\" (UniqueName: \"kubernetes.io/projected/afc1c28a-abd2-4def-a6d7-201732ee1eee-kube-api-access-ljq5b\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:44.779794 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.779749    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:44.779794 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.779763    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc1c28a-abd2-4def-a6d7-201732ee1eee-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:44.780767 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:44.780728    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:45.048705 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.048615    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:45.315205 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.314138    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:45.521163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.515915    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"]
Oct 30 18:59:45.521163 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.516359    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc" podUID="6153848e-01ea-40f0-ad84-70ebd26a4d32" containerName="payload-processing" containerID="cri-o://a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc" gracePeriod=30
Oct 30 18:59:45.522294 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.522238    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"]
Oct 30 18:59:45.522504 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.522476    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx" podUID="da3032fe-24ff-41ec-8798-93a6ac6a0420" containerName="payload-pre-processing" containerID="cri-o://1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a" gracePeriod=30
Oct 30 18:59:45.586336 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.584797    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:45.589965 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.589932    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f" event={"ID":"afc1c28a-abd2-4def-a6d7-201732ee1eee","Type":"ContainerDied","Data":"593a333c714f66699771305faab6b39bd49d507eff9194c3c3e86f0242497c5e"}
Oct 30 18:59:45.590143 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.589995    4925 scope.go:122] "RemoveContainer" containerID="7d19164f7c7c6b3bcd5ce69664ac0775acf77bd4e64f5e8495905a236b49083f"
Oct 30 18:59:45.590758 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.590603    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"
Oct 30 18:59:45.675668 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.675634    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"]
Oct 30 18:59:45.680367 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.680325    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-93574a-d4b5df766-54p4f"]
Oct 30 18:59:45.773215 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.773178    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:45.995130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:45.982769    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:46.351218 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.351189    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"
Oct 30 18:59:46.443631 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.443592    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afc1c28a-abd2-4def-a6d7-201732ee1eee" path="/var/lib/kubelet/pods/afc1c28a-abd2-4def-a6d7-201732ee1eee/volumes"
Oct 30 18:59:46.457386 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.457367    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"
Oct 30 18:59:46.480737 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.480692    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:46.546691 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.546617    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6153848e-01ea-40f0-ad84-70ebd26a4d32-kube-api-access-pg45p\" (UniqueName: \"kubernetes.io/projected/6153848e-01ea-40f0-ad84-70ebd26a4d32-kube-api-access-pg45p\") pod \"6153848e-01ea-40f0-ad84-70ebd26a4d32\" (UID: \"6153848e-01ea-40f0-ad84-70ebd26a4d32\") "
Oct 30 18:59:46.546861 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.546802    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6153848e-01ea-40f0-ad84-70ebd26a4d32-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6153848e-01ea-40f0-ad84-70ebd26a4d32-plugins-config-volume\") pod \"6153848e-01ea-40f0-ad84-70ebd26a4d32\" (UID: \"6153848e-01ea-40f0-ad84-70ebd26a4d32\") "
Oct 30 18:59:46.548451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.548420    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6153848e-01ea-40f0-ad84-70ebd26a4d32-plugins-config-volume" pod "6153848e-01ea-40f0-ad84-70ebd26a4d32" (UID: "6153848e-01ea-40f0-ad84-70ebd26a4d32"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:46.549766 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.549740    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6153848e-01ea-40f0-ad84-70ebd26a4d32-kube-api-access-pg45p" pod "6153848e-01ea-40f0-ad84-70ebd26a4d32" (UID: "6153848e-01ea-40f0-ad84-70ebd26a4d32"). InnerVolumeSpecName "kube-api-access-pg45p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:46.599465 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.599439    4925 generic.go:356] "Generic (PLEG): container finished" podID="6153848e-01ea-40f0-ad84-70ebd26a4d32" containerID="a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc" exitCode=0
Oct 30 18:59:46.599600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.599487    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"
Oct 30 18:59:46.599600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.599531    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc" event={"ID":"6153848e-01ea-40f0-ad84-70ebd26a4d32","Type":"ContainerDied","Data":"a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc"}
Oct 30 18:59:46.599600 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.599584    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc" event={"ID":"6153848e-01ea-40f0-ad84-70ebd26a4d32","Type":"ContainerDied","Data":"0e72ecdc2577fc635c0db60279b195a042b525eefa7709acf91160fe60f6c378"}
Oct 30 18:59:46.599732 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.599610    4925 scope.go:122] "RemoveContainer" containerID="a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc"
Oct 30 18:59:46.602717 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.602693    4925 generic.go:356] "Generic (PLEG): container finished" podID="da3032fe-24ff-41ec-8798-93a6ac6a0420" containerID="1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a" exitCode=0
Oct 30 18:59:46.602837 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.602738    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx" event={"ID":"da3032fe-24ff-41ec-8798-93a6ac6a0420","Type":"ContainerDied","Data":"1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a"}
Oct 30 18:59:46.602837 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.602774    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"
Oct 30 18:59:46.602837 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.602786    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx" event={"ID":"da3032fe-24ff-41ec-8798-93a6ac6a0420","Type":"ContainerDied","Data":"44a39116a39850ca26380d6c328386ce510ecee3b08cac3201b4a1fef675e3c9"}
Oct 30 18:59:46.612079 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.611951    4925 scope.go:122] "RemoveContainer" containerID="a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc"
Oct 30 18:59:46.612302 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:46.612283    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc\": container with ID starting with a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc not found: ID does not exist" containerID="a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc"
Oct 30 18:59:46.612353 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.612312    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc"} err="failed to get container status \"a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc\": rpc error: code = NotFound desc = could not find container \"a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc\": container with ID starting with a60043193e4b5cb81db7539a18aeb00fe31c7483d1bf8d0b467411c795b024fc not found: ID does not exist"
Oct 30 18:59:46.612353 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.612329    4925 scope.go:122] "RemoveContainer" containerID="1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a"
Oct 30 18:59:46.622631 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.622608    4925 scope.go:122] "RemoveContainer" containerID="1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a"
Oct 30 18:59:46.622896 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:46.622876    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a\": container with ID starting with 1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a not found: ID does not exist" containerID="1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a"
Oct 30 18:59:46.622970 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.622908    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a"} err="failed to get container status \"1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a\": rpc error: code = NotFound desc = could not find container \"1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a\": container with ID starting with 1ec896badc88409a3791b3c27f586842cab91125624896b648d7ef8f1f772e9a not found: ID does not exist"
Oct 30 18:59:46.625837 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.625816    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"]
Oct 30 18:59:46.637643 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.637606    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc"]
Oct 30 18:59:46.648597 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.648566    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da3032fe-24ff-41ec-8798-93a6ac6a0420-kube-api-access-rw2zh\" (UniqueName: \"kubernetes.io/projected/da3032fe-24ff-41ec-8798-93a6ac6a0420-kube-api-access-rw2zh\") pod \"da3032fe-24ff-41ec-8798-93a6ac6a0420\" (UID: \"da3032fe-24ff-41ec-8798-93a6ac6a0420\") "
Oct 30 18:59:46.648714 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.648694    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/da3032fe-24ff-41ec-8798-93a6ac6a0420-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3032fe-24ff-41ec-8798-93a6ac6a0420-plugins-config-volume\") pod \"da3032fe-24ff-41ec-8798-93a6ac6a0420\" (UID: \"da3032fe-24ff-41ec-8798-93a6ac6a0420\") "
Oct 30 18:59:46.649467 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.649418    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6153848e-01ea-40f0-ad84-70ebd26a4d32-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:46.649741 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.649465    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg45p\" (UniqueName: \"kubernetes.io/projected/6153848e-01ea-40f0-ad84-70ebd26a4d32-kube-api-access-pg45p\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:46.649954 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.649925    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da3032fe-24ff-41ec-8798-93a6ac6a0420-plugins-config-volume" pod "da3032fe-24ff-41ec-8798-93a6ac6a0420" (UID: "da3032fe-24ff-41ec-8798-93a6ac6a0420"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:46.651807 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.651576    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da3032fe-24ff-41ec-8798-93a6ac6a0420-kube-api-access-rw2zh" pod "da3032fe-24ff-41ec-8798-93a6ac6a0420" (UID: "da3032fe-24ff-41ec-8798-93a6ac6a0420"). InnerVolumeSpecName "kube-api-access-rw2zh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:46.662871 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.662843    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:46.755923 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.751318    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3032fe-24ff-41ec-8798-93a6ac6a0420-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:46.755923 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.751360    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw2zh\" (UniqueName: \"kubernetes.io/projected/da3032fe-24ff-41ec-8798-93a6ac6a0420-kube-api-access-rw2zh\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:46.925866 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.925820    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"]
Oct 30 18:59:46.932837 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.932808    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"]
Oct 30 18:59:46.944994 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:46.940302    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fe0612-7f874c54bc-jqzvx"]
Oct 30 18:59:47.096364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:47.096312    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:47.296026 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:47.295850    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:47.311696 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:47.311657    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-ipp-b-fe0612-586d4f84f6-cvhrc" podUID="6153848e-01ea-40f0-ad84-70ebd26a4d32" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.151:9004: i/o timeout"
Oct 30 18:59:47.712158 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:47.711300    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:48.491848 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.491668    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:48.520753 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.520667    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6153848e-01ea-40f0-ad84-70ebd26a4d32" path="/var/lib/kubelet/pods/6153848e-01ea-40f0-ad84-70ebd26a4d32/volumes"
Oct 30 18:59:48.522327 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.522032    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="da3032fe-24ff-41ec-8798-93a6ac6a0420" path="/var/lib/kubelet/pods/da3032fe-24ff-41ec-8798-93a6ac6a0420/volumes"
Oct 30 18:59:48.893829 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.893803    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:59:48.930033 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.929930    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-istio-token\" (UniqueName: \"kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-istio-token\") pod \"415056de-18a9-46fc-97cd-db55c15ce521\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") "
Oct 30 18:59:48.930033 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.929997    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/415056de-18a9-46fc-97cd-db55c15ce521-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/415056de-18a9-46fc-97cd-db55c15ce521-istiod-ca-cert\") pod \"415056de-18a9-46fc-97cd-db55c15ce521\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") "
Oct 30 18:59:48.930342 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.930047    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-certs\") pod \"415056de-18a9-46fc-97cd-db55c15ce521\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") "
Oct 30 18:59:48.930342 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.930183    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-kube-api-access-rkl8b\" (UniqueName: \"kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-kube-api-access-rkl8b\") pod \"415056de-18a9-46fc-97cd-db55c15ce521\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") "
Oct 30 18:59:48.930342 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.930236    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-credential-socket\") pod \"415056de-18a9-46fc-97cd-db55c15ce521\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") "
Oct 30 18:59:48.930457 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.930350    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-socket\") pod \"415056de-18a9-46fc-97cd-db55c15ce521\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") "
Oct 30 18:59:48.930457 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.930413    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-data\") pod \"415056de-18a9-46fc-97cd-db55c15ce521\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") "
Oct 30 18:59:48.930609 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.930590    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-envoy\") pod \"415056de-18a9-46fc-97cd-db55c15ce521\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") "
Oct 30 18:59:48.930707 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.930631    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/415056de-18a9-46fc-97cd-db55c15ce521-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/415056de-18a9-46fc-97cd-db55c15ce521-istio-podinfo\") pod \"415056de-18a9-46fc-97cd-db55c15ce521\" (UID: \"415056de-18a9-46fc-97cd-db55c15ce521\") "
Oct 30 18:59:48.930707 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.930647    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/415056de-18a9-46fc-97cd-db55c15ce521-istiod-ca-cert" pod "415056de-18a9-46fc-97cd-db55c15ce521" (UID: "415056de-18a9-46fc-97cd-db55c15ce521"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:48.931301 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.931276    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/415056de-18a9-46fc-97cd-db55c15ce521-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:48.931301 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.931275    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-certs" pod "415056de-18a9-46fc-97cd-db55c15ce521" (UID: "415056de-18a9-46fc-97cd-db55c15ce521"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:48.931467 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.931319    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-credential-socket" pod "415056de-18a9-46fc-97cd-db55c15ce521" (UID: "415056de-18a9-46fc-97cd-db55c15ce521"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:48.931552 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.931530    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-socket" pod "415056de-18a9-46fc-97cd-db55c15ce521" (UID: "415056de-18a9-46fc-97cd-db55c15ce521"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:48.933511 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.933452    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-data" pod "415056de-18a9-46fc-97cd-db55c15ce521" (UID: "415056de-18a9-46fc-97cd-db55c15ce521"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:48.933657 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.933602    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-istio-token" pod "415056de-18a9-46fc-97cd-db55c15ce521" (UID: "415056de-18a9-46fc-97cd-db55c15ce521"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:48.935227 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.935206    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/415056de-18a9-46fc-97cd-db55c15ce521-istio-podinfo" pod "415056de-18a9-46fc-97cd-db55c15ce521" (UID: "415056de-18a9-46fc-97cd-db55c15ce521"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:59:48.935227 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.935220    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-envoy" pod "415056de-18a9-46fc-97cd-db55c15ce521" (UID: "415056de-18a9-46fc-97cd-db55c15ce521"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:48.935971 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:48.935942    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-kube-api-access-rkl8b" pod "415056de-18a9-46fc-97cd-db55c15ce521" (UID: "415056de-18a9-46fc-97cd-db55c15ce521"). InnerVolumeSpecName "kube-api-access-rkl8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:49.032438 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.032364    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkl8b\" (UniqueName: \"kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-kube-api-access-rkl8b\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:49.032438 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.032394    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:49.032438 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.032404    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:49.032438 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.032413    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:49.032438 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.032421    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:49.032438 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.032429    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/415056de-18a9-46fc-97cd-db55c15ce521-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:49.032438 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.032437    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/415056de-18a9-46fc-97cd-db55c15ce521-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:49.032438 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.032445    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/415056de-18a9-46fc-97cd-db55c15ce521-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:49.267771 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.267734    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:49.628747 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.628709    4925 generic.go:356] "Generic (PLEG): container finished" podID="415056de-18a9-46fc-97cd-db55c15ce521" containerID="fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a" exitCode=0
Oct 30 18:59:49.629227 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.628758    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"
Oct 30 18:59:49.629227 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.628773    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" event={"ID":"415056de-18a9-46fc-97cd-db55c15ce521","Type":"ContainerDied","Data":"fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a"}
Oct 30 18:59:49.629227 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.628830    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv" event={"ID":"415056de-18a9-46fc-97cd-db55c15ce521","Type":"ContainerDied","Data":"d174970ea02168ec92d18d818a4eaf1b7c55b7251f87a1f025961871b0f1d0d1"}
Oct 30 18:59:49.629227 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.628863    4925 scope.go:122] "RemoveContainer" containerID="fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a"
Oct 30 18:59:49.643494 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.643474    4925 scope.go:122] "RemoveContainer" containerID="9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7"
Oct 30 18:59:49.654181 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.654136    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"]
Oct 30 18:59:49.655604 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.655578    4925 scope.go:122] "RemoveContainer" containerID="fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a"
Oct 30 18:59:49.655933 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:49.655897    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a\": container with ID starting with fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a not found: ID does not exist" containerID="fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a"
Oct 30 18:59:49.656009 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.655939    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a"} err="failed to get container status \"fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a\": rpc error: code = NotFound desc = could not find container \"fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a\": container with ID starting with fac0095d185cc089a760e92ec688ef81f13dae624c8d4f0d9704d17c2931ce1a not found: ID does not exist"
Oct 30 18:59:49.656009 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.655958    4925 scope.go:122] "RemoveContainer" containerID="9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7"
Oct 30 18:59:49.658203 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:49.656256    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7\": container with ID starting with 9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7 not found: ID does not exist" containerID="9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7"
Oct 30 18:59:49.658203 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.656290    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7"} err="failed to get container status \"9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7\": rpc error: code = NotFound desc = could not find container \"9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7\": container with ID starting with 9e75983efc4eaf55be48f30e9b10324e580e5f203550baa035274dfb7e087ac7 not found: ID does not exist"
Oct 30 18:59:49.658496 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.658476    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-655193-openshift-default-7fc4b9c797-wtfmv"]
Oct 30 18:59:49.890490 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:49.890346    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:50.178557 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:50.178521    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"]
Oct 30 18:59:50.340801 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:50.340771    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:50.423773 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:50.423729    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="415056de-18a9-46fc-97cd-db55c15ce521" path="/var/lib/kubelet/pods/415056de-18a9-46fc-97cd-db55c15ce521/volumes"
Oct 30 18:59:50.580170 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:50.578988    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:50.955144 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:50.955063    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"]
Oct 30 18:59:50.955640 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:50.955398    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw" podUID="f00d2ad1-ff8f-481c-82cf-74f20531ddde" containerName="maas-api" containerID="cri-o://9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395" gracePeriod=30
Oct 30 18:59:51.091343 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.091312    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:51.417134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.414886    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:51.440148 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.436520    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:59:51.594622 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.594527    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f00d2ad1-ff8f-481c-82cf-74f20531ddde-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f00d2ad1-ff8f-481c-82cf-74f20531ddde-openshift-service-ca\") pod \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") "
Oct 30 18:59:51.594622 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.594587    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-maas-api-tls\") pod \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") "
Oct 30 18:59:51.594881 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.594742    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f00d2ad1-ff8f-481c-82cf-74f20531ddde-kube-api-access-nkrb5\" (UniqueName: \"kubernetes.io/projected/f00d2ad1-ff8f-481c-82cf-74f20531ddde-kube-api-access-nkrb5\") pod \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") "
Oct 30 18:59:51.594881 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.594804    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-metrics-certs\") pod \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\" (UID: \"f00d2ad1-ff8f-481c-82cf-74f20531ddde\") "
Oct 30 18:59:51.596397 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.596371    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f00d2ad1-ff8f-481c-82cf-74f20531ddde-openshift-service-ca" pod "f00d2ad1-ff8f-481c-82cf-74f20531ddde" (UID: "f00d2ad1-ff8f-481c-82cf-74f20531ddde"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:51.597931 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.597907    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-maas-api-tls" pod "f00d2ad1-ff8f-481c-82cf-74f20531ddde" (UID: "f00d2ad1-ff8f-481c-82cf-74f20531ddde"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:59:51.598344 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.598326    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-metrics-certs" pod "f00d2ad1-ff8f-481c-82cf-74f20531ddde" (UID: "f00d2ad1-ff8f-481c-82cf-74f20531ddde"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:59:51.598774 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.598750    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f00d2ad1-ff8f-481c-82cf-74f20531ddde-kube-api-access-nkrb5" pod "f00d2ad1-ff8f-481c-82cf-74f20531ddde" (UID: "f00d2ad1-ff8f-481c-82cf-74f20531ddde"). InnerVolumeSpecName "kube-api-access-nkrb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:51.609380 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.609347    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:51.645172 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.645141    4925 generic.go:356] "Generic (PLEG): container finished" podID="f00d2ad1-ff8f-481c-82cf-74f20531ddde" containerID="9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395" exitCode=0
Oct 30 18:59:51.645297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.645202    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"
Oct 30 18:59:51.645297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.645221    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw" event={"ID":"f00d2ad1-ff8f-481c-82cf-74f20531ddde","Type":"ContainerDied","Data":"9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395"}
Oct 30 18:59:51.645297 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.645287    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw" event={"ID":"f00d2ad1-ff8f-481c-82cf-74f20531ddde","Type":"ContainerDied","Data":"77f3f98fd59308b34be15c021bab15e1fe6c84bf598944f83295e911b874ca06"}
Oct 30 18:59:51.645471 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.645318    4925 scope.go:122] "RemoveContainer" containerID="9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395"
Oct 30 18:59:51.661024 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.661005    4925 scope.go:122] "RemoveContainer" containerID="9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395"
Oct 30 18:59:51.661304 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:51.661283    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395\": container with ID starting with 9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395 not found: ID does not exist" containerID="9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395"
Oct 30 18:59:51.661383 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.661311    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395"} err="failed to get container status \"9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395\": rpc error: code = NotFound desc = could not find container \"9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395\": container with ID starting with 9451674c050df12d69cb0a16900efde2f4cda14dc938751a5481559e30f15395 not found: ID does not exist"
Oct 30 18:59:51.672934 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.672907    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"]
Oct 30 18:59:51.677965 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.677940    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f778f5-5b4886b88-sw7mw"]
Oct 30 18:59:51.697334 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.697304    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:51.697334 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.697333    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkrb5\" (UniqueName: \"kubernetes.io/projected/f00d2ad1-ff8f-481c-82cf-74f20531ddde-kube-api-access-nkrb5\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:51.697505 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.697351    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f00d2ad1-ff8f-481c-82cf-74f20531ddde-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:51.697505 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.697364    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f00d2ad1-ff8f-481c-82cf-74f20531ddde-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:51.807746 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.807706    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:51.927102 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:51.927068    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t" podUID="4cb3fee4-1878-4b77-8ee6-ec9767749621" containerName="payload-processing" containerID="cri-o://7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8" gracePeriod=25
Oct 30 18:59:52.235776 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.235753    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"
Oct 30 18:59:52.388134 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.376881    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:52.430048 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.430011    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f00d2ad1-ff8f-481c-82cf-74f20531ddde" path="/var/lib/kubelet/pods/f00d2ad1-ff8f-481c-82cf-74f20531ddde/volumes"
Oct 30 18:59:52.448973 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.448941    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4cb3fee4-1878-4b77-8ee6-ec9767749621-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb3fee4-1878-4b77-8ee6-ec9767749621-plugins-config-volume\") pod \"4cb3fee4-1878-4b77-8ee6-ec9767749621\" (UID: \"4cb3fee4-1878-4b77-8ee6-ec9767749621\") "
Oct 30 18:59:52.449215 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.449039    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cb3fee4-1878-4b77-8ee6-ec9767749621-kube-api-access-pjnwg\" (UniqueName: \"kubernetes.io/projected/4cb3fee4-1878-4b77-8ee6-ec9767749621-kube-api-access-pjnwg\") pod \"4cb3fee4-1878-4b77-8ee6-ec9767749621\" (UID: \"4cb3fee4-1878-4b77-8ee6-ec9767749621\") "
Oct 30 18:59:52.450037 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.450017    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cb3fee4-1878-4b77-8ee6-ec9767749621-plugins-config-volume" pod "4cb3fee4-1878-4b77-8ee6-ec9767749621" (UID: "4cb3fee4-1878-4b77-8ee6-ec9767749621"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:52.460332 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.460267    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cb3fee4-1878-4b77-8ee6-ec9767749621-kube-api-access-pjnwg" pod "4cb3fee4-1878-4b77-8ee6-ec9767749621" (UID: "4cb3fee4-1878-4b77-8ee6-ec9767749621"). InnerVolumeSpecName "kube-api-access-pjnwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:52.550885 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.550851    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb3fee4-1878-4b77-8ee6-ec9767749621-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:52.550885 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.550883    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjnwg\" (UniqueName: \"kubernetes.io/projected/4cb3fee4-1878-4b77-8ee6-ec9767749621-kube-api-access-pjnwg\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:52.698284 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.697692    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:52.703627 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.703439    4925 generic.go:356] "Generic (PLEG): container finished" podID="4cb3fee4-1878-4b77-8ee6-ec9767749621" containerID="7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8" exitCode=0
Oct 30 18:59:52.703627 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.703497    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"
Oct 30 18:59:52.703627 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.703527    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t" event={"ID":"4cb3fee4-1878-4b77-8ee6-ec9767749621","Type":"ContainerDied","Data":"7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8"}
Oct 30 18:59:52.703627 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.703601    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t" event={"ID":"4cb3fee4-1878-4b77-8ee6-ec9767749621","Type":"ContainerDied","Data":"d29ab6ef985f3f533ebdb35d74246453188e5ec89b1210f2f498520677620daf"}
Oct 30 18:59:52.703627 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.703633    4925 scope.go:122] "RemoveContainer" containerID="7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8"
Oct 30 18:59:52.718562 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.718545    4925 scope.go:122] "RemoveContainer" containerID="7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8"
Oct 30 18:59:52.718849 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:52.718829    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8\": container with ID starting with 7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8 not found: ID does not exist" containerID="7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8"
Oct 30 18:59:52.718924 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.718859    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8"} err="failed to get container status \"7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8\": rpc error: code = NotFound desc = could not find container \"7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8\": container with ID starting with 7343047b4c9e551895994029d019e94b7ec9ef28b51fad548122165c9e686ff8 not found: ID does not exist"
Oct 30 18:59:52.736936 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.736913    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"]
Oct 30 18:59:52.744137 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.744108    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t"]
Oct 30 18:59:52.898571 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.898536    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"]
Oct 30 18:59:52.898888 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.898778    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy" containerID="cri-o://bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002" gracePeriod=30
Oct 30 18:59:52.902364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:52.902341    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:53.109080 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:53.109001    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:53.134891 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:53.134841    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-f778f5-7fb6467b98-lfp5t" podUID="4cb3fee4-1878-4b77-8ee6-ec9767749621" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.145:50052\" within 1s: context deadline exceeded"
Oct 30 18:59:53.135680 ip-10-0-134-58 kubenswrapper[4925]: W1030 18:59:53.135214    4925 logging.go:55] [core] [Channel #2156 SubChannel #2157]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.145:50052", ServerName: "10.132.0.145:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.145:50052: operation was canceled"
Oct 30 18:59:53.874120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:53.870978    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:54.148474 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:54.148381    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"]
Oct 30 18:59:54.169839 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:54.166947    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"]
Oct 30 18:59:54.308481 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:54.308443    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:54.468130 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:54.456803    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4cb3fee4-1878-4b77-8ee6-ec9767749621" path="/var/lib/kubelet/pods/4cb3fee4-1878-4b77-8ee6-ec9767749621/volumes"
Oct 30 18:59:54.523135 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:54.520751    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:54.754720 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:54.753184    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:55.781165 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:55.779682    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:56.469138 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:56.467381    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:57.085869 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:57.083951    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:57.817664 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:57.817629    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:58.100457 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.098437    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:58.302155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.296963    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:58.302155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.300906    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"]
Oct 30 18:59:58.302155 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.301083    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy" containerID="cri-o://6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9" gracePeriod=30
Oct 30 18:59:58.513120 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.510163    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:58.527836 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.527800    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:59:58.622981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.622891    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-socket\") pod \"0a67c842-7520-4af1-86b3-ef20d5bb6149\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") "
Oct 30 18:59:58.622981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.622927    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-podinfo\") pod \"0a67c842-7520-4af1-86b3-ef20d5bb6149\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") "
Oct 30 18:59:58.622981 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.622972    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-credential-socket\") pod \"0a67c842-7520-4af1-86b3-ef20d5bb6149\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") "
Oct 30 18:59:58.623286 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.623057    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a67c842-7520-4af1-86b3-ef20d5bb6149-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a67c842-7520-4af1-86b3-ef20d5bb6149-istiod-ca-cert\") pod \"0a67c842-7520-4af1-86b3-ef20d5bb6149\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") "
Oct 30 18:59:58.623286 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.623125    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-data\") pod \"0a67c842-7520-4af1-86b3-ef20d5bb6149\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") "
Oct 30 18:59:58.623286 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.623184    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-certs\") pod \"0a67c842-7520-4af1-86b3-ef20d5bb6149\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") "
Oct 30 18:59:58.623286 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.623244    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-socket" pod "0a67c842-7520-4af1-86b3-ef20d5bb6149" (UID: "0a67c842-7520-4af1-86b3-ef20d5bb6149"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:58.623286 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.623259    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-envoy\") pod \"0a67c842-7520-4af1-86b3-ef20d5bb6149\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") "
Oct 30 18:59:58.623560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.623350    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-kube-api-access-qjcgv\" (UniqueName: \"kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-kube-api-access-qjcgv\") pod \"0a67c842-7520-4af1-86b3-ef20d5bb6149\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") "
Oct 30 18:59:58.623560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.623398    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-token\" (UniqueName: \"kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-token\") pod \"0a67c842-7520-4af1-86b3-ef20d5bb6149\" (UID: \"0a67c842-7520-4af1-86b3-ef20d5bb6149\") "
Oct 30 18:59:58.623560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.623485    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-certs" pod "0a67c842-7520-4af1-86b3-ef20d5bb6149" (UID: "0a67c842-7520-4af1-86b3-ef20d5bb6149"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:58.623560 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.623493    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-credential-socket" pod "0a67c842-7520-4af1-86b3-ef20d5bb6149" (UID: "0a67c842-7520-4af1-86b3-ef20d5bb6149"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:58.623827 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.623800    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a67c842-7520-4af1-86b3-ef20d5bb6149-istiod-ca-cert" pod "0a67c842-7520-4af1-86b3-ef20d5bb6149" (UID: "0a67c842-7520-4af1-86b3-ef20d5bb6149"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:58.624110 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.624042    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-data" pod "0a67c842-7520-4af1-86b3-ef20d5bb6149" (UID: "0a67c842-7520-4af1-86b3-ef20d5bb6149"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:58.624110 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.624042    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:58.624110 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.624123    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:58.624110 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.624144    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:58.624110 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.624160    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a67c842-7520-4af1-86b3-ef20d5bb6149-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:58.625784 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.625758    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-envoy" pod "0a67c842-7520-4af1-86b3-ef20d5bb6149" (UID: "0a67c842-7520-4af1-86b3-ef20d5bb6149"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:59:58.626006 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.625975    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-token" pod "0a67c842-7520-4af1-86b3-ef20d5bb6149" (UID: "0a67c842-7520-4af1-86b3-ef20d5bb6149"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:58.626411 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.626389    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-kube-api-access-qjcgv" pod "0a67c842-7520-4af1-86b3-ef20d5bb6149" (UID: "0a67c842-7520-4af1-86b3-ef20d5bb6149"). InnerVolumeSpecName "kube-api-access-qjcgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:58.626716 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.626692    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-podinfo" pod "0a67c842-7520-4af1-86b3-ef20d5bb6149" (UID: "0a67c842-7520-4af1-86b3-ef20d5bb6149"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:59:58.725451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.725422    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:58.725451 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.725448    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:58.725672 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.725458    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:58.725672 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.725469    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjcgv\" (UniqueName: \"kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-kube-api-access-qjcgv\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:58.725672 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.725478    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a67c842-7520-4af1-86b3-ef20d5bb6149-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:58.818122 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.817351    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:58.829763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.828572    4925 generic.go:356] "Generic (PLEG): container finished" podID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerID="bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002" exitCode=0
Oct 30 18:59:58.829763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.828640    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" event={"ID":"0a67c842-7520-4af1-86b3-ef20d5bb6149","Type":"ContainerDied","Data":"bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002"}
Oct 30 18:59:58.829763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.828688    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"
Oct 30 18:59:58.829763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.828698    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl" event={"ID":"0a67c842-7520-4af1-86b3-ef20d5bb6149","Type":"ContainerDied","Data":"3ac3e74c56f2da37eccb62a1caaac8fbe14f243bbb43e2eb9e8a774f9097c871"}
Oct 30 18:59:58.829763 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.828730    4925 scope.go:122] "RemoveContainer" containerID="bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002"
Oct 30 18:59:58.842552 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.842531    4925 scope.go:122] "RemoveContainer" containerID="59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d"
Oct 30 18:59:58.855604 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.855574    4925 scope.go:122] "RemoveContainer" containerID="bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002"
Oct 30 18:59:58.855968 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:58.855939    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002\": container with ID starting with bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002 not found: ID does not exist" containerID="bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002"
Oct 30 18:59:58.856117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.855977    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002"} err="failed to get container status \"bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002\": rpc error: code = NotFound desc = could not find container \"bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002\": container with ID starting with bc8e286795f285c7d6f4cedadf2a97f8508f974b9b7456b2c5c8a44024c23002 not found: ID does not exist"
Oct 30 18:59:58.856117 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.856003    4925 scope.go:122] "RemoveContainer" containerID="59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d"
Oct 30 18:59:58.857081 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:58.857054    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d\": container with ID starting with 59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d not found: ID does not exist" containerID="59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d"
Oct 30 18:59:58.857209 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.857160    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d"} err="failed to get container status \"59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d\": rpc error: code = NotFound desc = could not find container \"59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d\": container with ID starting with 59b24b7c1baf7553bad3316022437116be5529335de642c8503e731b5374aa0d not found: ID does not exist"
Oct 30 18:59:58.862216 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.862189    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"]
Oct 30 18:59:58.864704 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:58.864680    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-fe0612-openshift-default-884c77ff7-q4fdl"]
Oct 30 18:59:59.149167 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.149124    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z" podUID="fae6cb79-cee1-48f6-aff1-29f3b3bff735" containerName="payload-processing" containerID="cri-o://eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9" gracePeriod=25
Oct 30 18:59:59.168131 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.168068    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q" podUID="89911b2c-ec95-4aa1-b6cc-5c712d5cabae" containerName="payload-pre-processing" containerID="cri-o://9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154" gracePeriod=25
Oct 30 18:59:59.499687 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.499660    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"
Oct 30 18:59:59.535993 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.535966    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fae6cb79-cee1-48f6-aff1-29f3b3bff735-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fae6cb79-cee1-48f6-aff1-29f3b3bff735-plugins-config-volume\") pod \"fae6cb79-cee1-48f6-aff1-29f3b3bff735\" (UID: \"fae6cb79-cee1-48f6-aff1-29f3b3bff735\") "
Oct 30 18:59:59.536177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.536131    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fae6cb79-cee1-48f6-aff1-29f3b3bff735-kube-api-access-8bntj\" (UniqueName: \"kubernetes.io/projected/fae6cb79-cee1-48f6-aff1-29f3b3bff735-kube-api-access-8bntj\") pod \"fae6cb79-cee1-48f6-aff1-29f3b3bff735\" (UID: \"fae6cb79-cee1-48f6-aff1-29f3b3bff735\") "
Oct 30 18:59:59.536400 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.536377    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fae6cb79-cee1-48f6-aff1-29f3b3bff735-plugins-config-volume" pod "fae6cb79-cee1-48f6-aff1-29f3b3bff735" (UID: "fae6cb79-cee1-48f6-aff1-29f3b3bff735"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:59.536593 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.536575    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fae6cb79-cee1-48f6-aff1-29f3b3bff735-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:59.538364 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.538325    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"
Oct 30 18:59:59.538870 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.538850    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fae6cb79-cee1-48f6-aff1-29f3b3bff735-kube-api-access-8bntj" pod "fae6cb79-cee1-48f6-aff1-29f3b3bff735" (UID: "fae6cb79-cee1-48f6-aff1-29f3b3bff735"). InnerVolumeSpecName "kube-api-access-8bntj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:59.637694 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.637663    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-plugins-config-volume\") pod \"89911b2c-ec95-4aa1-b6cc-5c712d5cabae\" (UID: \"89911b2c-ec95-4aa1-b6cc-5c712d5cabae\") "
Oct 30 18:59:59.637860 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.637761    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-kube-api-access-pqtsp\" (UniqueName: \"kubernetes.io/projected/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-kube-api-access-pqtsp\") pod \"89911b2c-ec95-4aa1-b6cc-5c712d5cabae\" (UID: \"89911b2c-ec95-4aa1-b6cc-5c712d5cabae\") "
Oct 30 18:59:59.638111 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.638069    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-plugins-config-volume" pod "89911b2c-ec95-4aa1-b6cc-5c712d5cabae" (UID: "89911b2c-ec95-4aa1-b6cc-5c712d5cabae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:59:59.638287 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.638269    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:59.638323 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.638294    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bntj\" (UniqueName: \"kubernetes.io/projected/fae6cb79-cee1-48f6-aff1-29f3b3bff735-kube-api-access-8bntj\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:59.640020 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.640002    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-kube-api-access-pqtsp" pod "89911b2c-ec95-4aa1-b6cc-5c712d5cabae" (UID: "89911b2c-ec95-4aa1-b6cc-5c712d5cabae"). InnerVolumeSpecName "kube-api-access-pqtsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:59:59.748237 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.748153    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqtsp\" (UniqueName: \"kubernetes.io/projected/89911b2c-ec95-4aa1-b6cc-5c712d5cabae-kube-api-access-pqtsp\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 18:59:59.864381 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.864331    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:59.886692 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.886659    4925 generic.go:356] "Generic (PLEG): container finished" podID="89911b2c-ec95-4aa1-b6cc-5c712d5cabae" containerID="9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154" exitCode=0
Oct 30 18:59:59.886859 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.886738    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"
Oct 30 18:59:59.886976 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.886953    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q" event={"ID":"89911b2c-ec95-4aa1-b6cc-5c712d5cabae","Type":"ContainerDied","Data":"9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154"}
Oct 30 18:59:59.887066 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.887042    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q" event={"ID":"89911b2c-ec95-4aa1-b6cc-5c712d5cabae","Type":"ContainerDied","Data":"9f05766928da13cdb04dcf50e75196dc1227d66905df658c6281c34b6df14402"}
Oct 30 18:59:59.887177 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.887077    4925 scope.go:122] "RemoveContainer" containerID="9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154"
Oct 30 18:59:59.925412 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.925375    4925 generic.go:356] "Generic (PLEG): container finished" podID="fae6cb79-cee1-48f6-aff1-29f3b3bff735" containerID="eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9" exitCode=0
Oct 30 18:59:59.925546 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.925471    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z" event={"ID":"fae6cb79-cee1-48f6-aff1-29f3b3bff735","Type":"ContainerDied","Data":"eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9"}
Oct 30 18:59:59.925546 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.925523    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z" event={"ID":"fae6cb79-cee1-48f6-aff1-29f3b3bff735","Type":"ContainerDied","Data":"faf236d3d5efa5259b108575f39daa408252a650fdd376edc7fe70cbb2c38e18"}
Oct 30 18:59:59.925646 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.925620    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"
Oct 30 18:59:59.928766 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.928737    4925 scope.go:122] "RemoveContainer" containerID="9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154"
Oct 30 18:59:59.929192 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:59.929161    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154\": container with ID starting with 9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154 not found: ID does not exist" containerID="9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154"
Oct 30 18:59:59.929351 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.929203    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154"} err="failed to get container status \"9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154\": rpc error: code = NotFound desc = could not find container \"9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154\": container with ID starting with 9c956ee1fd38788ef48e39ec594e62360260723459328d3a42955d3731715154 not found: ID does not exist"
Oct 30 18:59:59.929351 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.929228    4925 scope.go:122] "RemoveContainer" containerID="eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9"
Oct 30 18:59:59.942070 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.942039    4925 scope.go:122] "RemoveContainer" containerID="eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9"
Oct 30 18:59:59.942387 ip-10-0-134-58 kubenswrapper[4925]: E1030 18:59:59.942361    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9\": container with ID starting with eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9 not found: ID does not exist" containerID="eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9"
Oct 30 18:59:59.942488 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.942396    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9"} err="failed to get container status \"eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9\": rpc error: code = NotFound desc = could not find container \"eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9\": container with ID starting with eb62fbdb731f4243603be646ae459a62dd0fe582efb00d657841df34026ff2b9 not found: ID does not exist"
Oct 30 18:59:59.946384 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.945699    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"]
Oct 30 18:59:59.951988 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.951963    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-10d367-6cbf6ddcbc-xnr6q"]
Oct 30 18:59:59.966641 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.966613    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"]
Oct 30 18:59:59.969890 ip-10-0-134-58 kubenswrapper[4925]: I1030 18:59:59.969870    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-10d367-5fc56d84b4-zdh4z"]
Oct 30 19:00:00.184596 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.183545    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"]
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184628    4925 cpu_manager.go:414] "containerMap: removing container" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184658    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184677    4925 cpu_manager.go:414] "containerMap: removing container" podUID="fd35ca47-1812-4719-ad92-5993b466e8c2" containerName="payload-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184686    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd35ca47-1812-4719-ad92-5993b466e8c2" containerName="payload-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184719    4925 cpu_manager.go:414] "containerMap: removing container" podUID="89911b2c-ec95-4aa1-b6cc-5c712d5cabae" containerName="payload-pre-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184729    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89911b2c-ec95-4aa1-b6cc-5c712d5cabae" containerName="payload-pre-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184744    4925 cpu_manager.go:414] "containerMap: removing container" podUID="fae6cb79-cee1-48f6-aff1-29f3b3bff735" containerName="payload-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184752    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fae6cb79-cee1-48f6-aff1-29f3b3bff735" containerName="payload-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184776    4925 cpu_manager.go:414] "containerMap: removing container" podUID="f00d2ad1-ff8f-481c-82cf-74f20531ddde" containerName="maas-api"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184783    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f00d2ad1-ff8f-481c-82cf-74f20531ddde" containerName="maas-api"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184800    4925 cpu_manager.go:414] "containerMap: removing container" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184807    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184826    4925 cpu_manager.go:414] "containerMap: removing container" podUID="0bef6ca6-3659-49e8-8fa7-b3984d0be9a4" containerName="payload-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184833    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0bef6ca6-3659-49e8-8fa7-b3984d0be9a4" containerName="payload-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184849    4925 cpu_manager.go:414] "containerMap: removing container" podUID="0f6a689f-9e21-473c-b918-e2a05e18c87f" containerName="payload-pre-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184856    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f6a689f-9e21-473c-b918-e2a05e18c87f" containerName="payload-pre-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184876    4925 cpu_manager.go:414] "containerMap: removing container" podUID="ad8ce530-37d0-48f6-805d-1b034492b4fe" containerName="payload-pre-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184883    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad8ce530-37d0-48f6-805d-1b034492b4fe" containerName="payload-pre-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184895    4925 cpu_manager.go:414] "containerMap: removing container" podUID="afc1c28a-abd2-4def-a6d7-201732ee1eee" containerName="maas-api"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184901    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afc1c28a-abd2-4def-a6d7-201732ee1eee" containerName="maas-api"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184919    4925 cpu_manager.go:414] "containerMap: removing container" podUID="4cb3fee4-1878-4b77-8ee6-ec9767749621" containerName="payload-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184926    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cb3fee4-1878-4b77-8ee6-ec9767749621" containerName="payload-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184937    4925 cpu_manager.go:414] "containerMap: removing container" podUID="6153848e-01ea-40f0-ad84-70ebd26a4d32" containerName="payload-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184944    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6153848e-01ea-40f0-ad84-70ebd26a4d32" containerName="payload-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184961    4925 cpu_manager.go:414] "containerMap: removing container" podUID="da3032fe-24ff-41ec-8798-93a6ac6a0420" containerName="payload-pre-processing"
Oct 30 19:00:00.185029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.184968    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da3032fe-24ff-41ec-8798-93a6ac6a0420" containerName="payload-pre-processing"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185141    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185155    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd35ca47-1812-4719-ad92-5993b466e8c2" containerName="payload-processing"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185174    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="89911b2c-ec95-4aa1-b6cc-5c712d5cabae" containerName="payload-pre-processing"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185184    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="fae6cb79-cee1-48f6-aff1-29f3b3bff735" containerName="payload-processing"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185194    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="f00d2ad1-ff8f-481c-82cf-74f20531ddde" containerName="maas-api"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185204    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185215    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="0bef6ca6-3659-49e8-8fa7-b3984d0be9a4" containerName="payload-processing"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185226    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f6a689f-9e21-473c-b918-e2a05e18c87f" containerName="payload-pre-processing"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185237    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad8ce530-37d0-48f6-805d-1b034492b4fe" containerName="payload-pre-processing"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185247    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="afc1c28a-abd2-4def-a6d7-201732ee1eee" containerName="maas-api"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185258    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cb3fee4-1878-4b77-8ee6-ec9767749621" containerName="payload-processing"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185268    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="6153848e-01ea-40f0-ad84-70ebd26a4d32" containerName="payload-processing"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185277    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy"
Oct 30 19:00:00.194299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.185288    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="da3032fe-24ff-41ec-8798-93a6ac6a0420" containerName="payload-pre-processing"
Oct 30 19:00:00.200146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.198752    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"]
Oct 30 19:00:00.200146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.198801    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"]
Oct 30 19:00:00.200146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.199770    4925 cpu_manager.go:414] "containerMap: removing container" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy"
Oct 30 19:00:00.200146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.199791    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy"
Oct 30 19:00:00.200146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.199837    4925 cpu_manager.go:414] "containerMap: removing container" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy"
Oct 30 19:00:00.200146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.199843    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="415056de-18a9-46fc-97cd-db55c15ce521" containerName="istio-proxy"
Oct 30 19:00:00.200146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.199791    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"
Oct 30 19:00:00.200668 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.200337    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" containerName="istio-proxy"
Oct 30 19:00:00.204429 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.204409    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"
Oct 30 19:00:00.216260 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.216233    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"]
Oct 30 19:00:00.260333 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.260307    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-94hcn\""
Oct 30 19:00:00.277051 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.277022    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtfcb\" (UniqueName: \"kubernetes.io/projected/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-kube-api-access-qtfcb\") pod \"maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd\" (UID: \"6ba5df2e-f25f-4ae2-8612-92d72aa3a520\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"
Oct 30 19:00:00.277209 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.277082    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6\" (UID: \"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"
Oct 30 19:00:00.277303 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.277281    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd\" (UID: \"6ba5df2e-f25f-4ae2-8612-92d72aa3a520\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"
Oct 30 19:00:00.277369 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.277327    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79d6w\" (UniqueName: \"kubernetes.io/projected/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-kube-api-access-79d6w\") pod \"maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6\" (UID: \"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"
Oct 30 19:00:00.378305 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.378271    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtfcb\" (UniqueName: \"kubernetes.io/projected/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-kube-api-access-qtfcb\") pod \"maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd\" (UID: \"6ba5df2e-f25f-4ae2-8612-92d72aa3a520\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"
Oct 30 19:00:00.378434 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.378318    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6\" (UID: \"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"
Oct 30 19:00:00.378494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.378441    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd\" (UID: \"6ba5df2e-f25f-4ae2-8612-92d72aa3a520\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"
Oct 30 19:00:00.378494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.378483    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79d6w\" (UniqueName: \"kubernetes.io/projected/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-kube-api-access-79d6w\") pod \"maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6\" (UID: \"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"
Oct 30 19:00:00.378963 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.378942    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6\" (UID: \"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"
Oct 30 19:00:00.379130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.379112    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd\" (UID: \"6ba5df2e-f25f-4ae2-8612-92d72aa3a520\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"
Oct 30 19:00:00.388572 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.388546    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79d6w\" (UniqueName: \"kubernetes.io/projected/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-kube-api-access-79d6w\") pod \"maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6\" (UID: \"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"
Oct 30 19:00:00.390547 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.390522    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtfcb\" (UniqueName: \"kubernetes.io/projected/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-kube-api-access-qtfcb\") pod \"maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd\" (UID: \"6ba5df2e-f25f-4ae2-8612-92d72aa3a520\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"
Oct 30 19:00:00.401334 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.401306    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"]
Oct 30 19:00:00.401706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.401659    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks" podUID="03a6f6b5-669d-449a-86b4-bbf8d0329f58" containerName="maas-api" containerID="cri-o://af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20" gracePeriod=30
Oct 30 19:00:00.435063 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.434989    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a67c842-7520-4af1-86b3-ef20d5bb6149" path="/var/lib/kubelet/pods/0a67c842-7520-4af1-86b3-ef20d5bb6149/volumes"
Oct 30 19:00:00.435663 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.435644    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89911b2c-ec95-4aa1-b6cc-5c712d5cabae" path="/var/lib/kubelet/pods/89911b2c-ec95-4aa1-b6cc-5c712d5cabae/volumes"
Oct 30 19:00:00.436239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.436216    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fae6cb79-cee1-48f6-aff1-29f3b3bff735" path="/var/lib/kubelet/pods/fae6cb79-cee1-48f6-aff1-29f3b3bff735/volumes"
Oct 30 19:00:00.535031 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.534998    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"
Oct 30 19:00:00.539523 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.538884    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:00.547879 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.547848    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"
Oct 30 19:00:00.764336 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.757751    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:00.910815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.910783    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 19:00:00.961147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.958581    4925 generic.go:356] "Generic (PLEG): container finished" podID="03a6f6b5-669d-449a-86b4-bbf8d0329f58" containerID="af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20" exitCode=0
Oct 30 19:00:00.961147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.958782    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"
Oct 30 19:00:00.961147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.959164    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks" event={"ID":"03a6f6b5-669d-449a-86b4-bbf8d0329f58","Type":"ContainerDied","Data":"af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20"}
Oct 30 19:00:00.961147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.959208    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:00.961147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.959229    4925 scope.go:122] "RemoveContainer" containerID="af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20"
Oct 30 19:00:00.961147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.959215    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks" event={"ID":"03a6f6b5-669d-449a-86b4-bbf8d0329f58","Type":"ContainerDied","Data":"f62b8d8e058c0e445fca499453cdb0dcb3d6d42fb9cad3ca1d6ada4525406e7b"}
Oct 30 19:00:00.977259 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.977233    4925 scope.go:122] "RemoveContainer" containerID="af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20"
Oct 30 19:00:00.980346 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:00.980316    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20\": container with ID starting with af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20 not found: ID does not exist" containerID="af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20"
Oct 30 19:00:00.980477 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:00.980356    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20"} err="failed to get container status \"af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20\": rpc error: code = NotFound desc = could not find container \"af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20\": container with ID starting with af7fcb5d80f0f231906065a97e646585324685354d78c1c61f4b1cc0df909f20 not found: ID does not exist"
Oct 30 19:00:01.000552 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.000531    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03a6f6b5-669d-449a-86b4-bbf8d0329f58-kube-api-access-p6cd5\" (UniqueName: \"kubernetes.io/projected/03a6f6b5-669d-449a-86b4-bbf8d0329f58-kube-api-access-p6cd5\") pod \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") "
Oct 30 19:00:01.000690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.000592    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03a6f6b5-669d-449a-86b4-bbf8d0329f58-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a6f6b5-669d-449a-86b4-bbf8d0329f58-openshift-service-ca\") pod \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") "
Oct 30 19:00:01.000690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.000641    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-maas-api-tls\") pod \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") "
Oct 30 19:00:01.000855 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.000836    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-metrics-certs\" (UniqueName: \"kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-metrics-certs\") pod \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\" (UID: \"03a6f6b5-669d-449a-86b4-bbf8d0329f58\") "
Oct 30 19:00:01.001015 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.000989    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03a6f6b5-669d-449a-86b4-bbf8d0329f58-openshift-service-ca" pod "03a6f6b5-669d-449a-86b4-bbf8d0329f58" (UID: "03a6f6b5-669d-449a-86b4-bbf8d0329f58"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:01.001496 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.001463    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03a6f6b5-669d-449a-86b4-bbf8d0329f58-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:01.002907 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.002882    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03a6f6b5-669d-449a-86b4-bbf8d0329f58-kube-api-access-p6cd5" pod "03a6f6b5-669d-449a-86b4-bbf8d0329f58" (UID: "03a6f6b5-669d-449a-86b4-bbf8d0329f58"). InnerVolumeSpecName "kube-api-access-p6cd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:01.003135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.003110    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-metrics-certs" pod "03a6f6b5-669d-449a-86b4-bbf8d0329f58" (UID: "03a6f6b5-669d-449a-86b4-bbf8d0329f58"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:00:01.003332 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.003314    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-maas-api-tls" pod "03a6f6b5-669d-449a-86b4-bbf8d0329f58" (UID: "03a6f6b5-669d-449a-86b4-bbf8d0329f58"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:00:01.039897 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.039867    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"]
Oct 30 19:00:01.047211 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:00:01.047156    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68ca6d6e_610a_463c_b383_f2c9c6f5a9cc.slice/crio-b680460df7fb5a463d6b65c07e0d30e9e07a8a4563f7f71a111217d18cc7e18a WatchSource:0}: Error finding container b680460df7fb5a463d6b65c07e0d30e9e07a8a4563f7f71a111217d18cc7e18a: Status 404 returned error can't find the container with id b680460df7fb5a463d6b65c07e0d30e9e07a8a4563f7f71a111217d18cc7e18a
Oct 30 19:00:01.073669 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.073627    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"]
Oct 30 19:00:01.122163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.120511    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:01.122163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.120563    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6cd5\" (UniqueName: \"kubernetes.io/projected/03a6f6b5-669d-449a-86b4-bbf8d0329f58-kube-api-access-p6cd5\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:01.122163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.120582    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03a6f6b5-669d-449a-86b4-bbf8d0329f58-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:01.153064 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.153032    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:01.290506 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.290408    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"]
Oct 30 19:00:01.295076 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.295046    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-10d367-74f886d88f-jb9ks"]
Oct 30 19:00:01.335535 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.335505    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:01.957110 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.957057    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:01.975672 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.975633    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd" event={"ID":"6ba5df2e-f25f-4ae2-8612-92d72aa3a520","Type":"ContainerStarted","Data":"044da68751a97e6d58c733c66806456c0f831670cee5cd35ea22f7ca996072c0"}
Oct 30 19:00:01.978112 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:01.978074    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6" event={"ID":"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc","Type":"ContainerStarted","Data":"b680460df7fb5a463d6b65c07e0d30e9e07a8a4563f7f71a111217d18cc7e18a"}
Oct 30 19:00:02.424872 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:02.424835    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03a6f6b5-669d-449a-86b4-bbf8d0329f58" path="/var/lib/kubelet/pods/03a6f6b5-669d-449a-86b4-bbf8d0329f58/volumes"
Oct 30 19:00:02.728173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:02.726479    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:03.494973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.494930    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:03.734391 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.734364    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 19:00:03.804901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.803458    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d699d780-921f-48ae-8ce4-27fdd806b52d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d699d780-921f-48ae-8ce4-27fdd806b52d-istiod-ca-cert\") pod \"d699d780-921f-48ae-8ce4-27fdd806b52d\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") "
Oct 30 19:00:03.804901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.803512    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-data\") pod \"d699d780-921f-48ae-8ce4-27fdd806b52d\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") "
Oct 30 19:00:03.804901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.803694    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-socket\") pod \"d699d780-921f-48ae-8ce4-27fdd806b52d\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") "
Oct 30 19:00:03.804901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.803761    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-certs\") pod \"d699d780-921f-48ae-8ce4-27fdd806b52d\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") "
Oct 30 19:00:03.804901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.803828    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-kube-api-access-lzpb4\" (UniqueName: \"kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-kube-api-access-lzpb4\") pod \"d699d780-921f-48ae-8ce4-27fdd806b52d\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") "
Oct 30 19:00:03.804901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.803894    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-envoy\") pod \"d699d780-921f-48ae-8ce4-27fdd806b52d\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") "
Oct 30 19:00:03.804901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.803921    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-podinfo\") pod \"d699d780-921f-48ae-8ce4-27fdd806b52d\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") "
Oct 30 19:00:03.804901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.803950    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-token\" (UniqueName: \"kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-token\") pod \"d699d780-921f-48ae-8ce4-27fdd806b52d\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") "
Oct 30 19:00:03.804901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.803986    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-credential-socket\") pod \"d699d780-921f-48ae-8ce4-27fdd806b52d\" (UID: \"d699d780-921f-48ae-8ce4-27fdd806b52d\") "
Oct 30 19:00:03.805615 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.805189    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-data" pod "d699d780-921f-48ae-8ce4-27fdd806b52d" (UID: "d699d780-921f-48ae-8ce4-27fdd806b52d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:03.805766 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.805709    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-socket" pod "d699d780-921f-48ae-8ce4-27fdd806b52d" (UID: "d699d780-921f-48ae-8ce4-27fdd806b52d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:03.805941 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.805921    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d699d780-921f-48ae-8ce4-27fdd806b52d-istiod-ca-cert" pod "d699d780-921f-48ae-8ce4-27fdd806b52d" (UID: "d699d780-921f-48ae-8ce4-27fdd806b52d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:03.806042 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.806018    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-certs" pod "d699d780-921f-48ae-8ce4-27fdd806b52d" (UID: "d699d780-921f-48ae-8ce4-27fdd806b52d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:03.806221 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.806126    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-credential-socket" pod "d699d780-921f-48ae-8ce4-27fdd806b52d" (UID: "d699d780-921f-48ae-8ce4-27fdd806b52d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:03.807912 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.807879    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-envoy" pod "d699d780-921f-48ae-8ce4-27fdd806b52d" (UID: "d699d780-921f-48ae-8ce4-27fdd806b52d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:03.808522 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.808494    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-podinfo" pod "d699d780-921f-48ae-8ce4-27fdd806b52d" (UID: "d699d780-921f-48ae-8ce4-27fdd806b52d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:03.808794 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.808759    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-token" pod "d699d780-921f-48ae-8ce4-27fdd806b52d" (UID: "d699d780-921f-48ae-8ce4-27fdd806b52d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:03.808874 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.808854    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-kube-api-access-lzpb4" pod "d699d780-921f-48ae-8ce4-27fdd806b52d" (UID: "d699d780-921f-48ae-8ce4-27fdd806b52d"). InnerVolumeSpecName "kube-api-access-lzpb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:03.906353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.906275    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:03.906353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.906304    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:03.906353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.906316    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzpb4\" (UniqueName: \"kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-kube-api-access-lzpb4\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:03.906353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.906330    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:03.906353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.906344    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:03.906353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.906354    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:03.906666 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.906365    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:03.906666 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.906374    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d699d780-921f-48ae-8ce4-27fdd806b52d-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:03.906666 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:03.906382    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d699d780-921f-48ae-8ce4-27fdd806b52d-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:04.001471 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.001437    4925 generic.go:356] "Generic (PLEG): container finished" podID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerID="6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9" exitCode=0
Oct 30 19:00:04.001706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.001618    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"
Oct 30 19:00:04.001706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.001677    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" event={"ID":"d699d780-921f-48ae-8ce4-27fdd806b52d","Type":"ContainerDied","Data":"6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9"}
Oct 30 19:00:04.001839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.001731    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw" event={"ID":"d699d780-921f-48ae-8ce4-27fdd806b52d","Type":"ContainerDied","Data":"776ff6f03a1675686f4cd1bb9220b04873740fb1ed639e2374da0642c888312d"}
Oct 30 19:00:04.001839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.001764    4925 scope.go:122] "RemoveContainer" containerID="6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9"
Oct 30 19:00:04.022329 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.022309    4925 scope.go:122] "RemoveContainer" containerID="07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858"
Oct 30 19:00:04.031531 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.031500    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"]
Oct 30 19:00:04.035839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.035815    4925 scope.go:122] "RemoveContainer" containerID="6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9"
Oct 30 19:00:04.036660 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:04.036241    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9\": container with ID starting with 6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9 not found: ID does not exist" containerID="6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9"
Oct 30 19:00:04.036660 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.036282    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9"} err="failed to get container status \"6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9\": rpc error: code = NotFound desc = could not find container \"6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9\": container with ID starting with 6daa9870865f0bd42d6cb23d4f62b21ef912f5bc8e61261ca84723cfdddb95e9 not found: ID does not exist"
Oct 30 19:00:04.036660 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.036310    4925 scope.go:122] "RemoveContainer" containerID="07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858"
Oct 30 19:00:04.036660 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.036492    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-f778f5-openshift-default-6d944754c7-xq8rw"]
Oct 30 19:00:04.036924 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:04.036665    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858\": container with ID starting with 07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858 not found: ID does not exist" containerID="07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858"
Oct 30 19:00:04.036924 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.036696    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858"} err="failed to get container status \"07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858\": rpc error: code = NotFound desc = could not find container \"07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858\": container with ID starting with 07150ff120010af38d151c5a7d9a4e102231b73ef9982c257677f679e220a858 not found: ID does not exist"
Oct 30 19:00:04.378663 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.378578    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:04.427688 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:04.427655    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" path="/var/lib/kubelet/pods/d699d780-921f-48ae-8ce4-27fdd806b52d/volumes"
Oct 30 19:00:05.209285 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.208720    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm" podUID="47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" containerName="main" containerID="cri-o://8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6" gracePeriod=45
Oct 30 19:00:05.210789 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.210375    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:05.523443 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.523375    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 19:00:05.636109 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.636066    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-kube-api-access-8mzhn\" (UniqueName: \"kubernetes.io/projected/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-kube-api-access-8mzhn\") pod \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") "
Oct 30 19:00:05.636255 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.636181    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-dshm\") pod \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") "
Oct 30 19:00:05.636319 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.636268    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tmp-dir\") pod \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") "
Oct 30 19:00:05.636374 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.636324    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tls-certs\" (UniqueName: \"kubernetes.io/secret/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tls-certs\") pod \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") "
Oct 30 19:00:05.636444 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.636388    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-home\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-home\") pod \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") "
Oct 30 19:00:05.636444 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.636427    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-model-cache\") pod \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\" (UID: \"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb\") "
Oct 30 19:00:05.637665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.636550    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tmp-dir" pod "47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" (UID: "47fba953-aa96-4b41-9cc5-f00d6c6b2ffb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:05.637806 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.637694    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-home" pod "47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" (UID: "47fba953-aa96-4b41-9cc5-f00d6c6b2ffb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:05.642330 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.642300    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-kube-api-access-8mzhn" pod "47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" (UID: "47fba953-aa96-4b41-9cc5-f00d6c6b2ffb"). InnerVolumeSpecName "kube-api-access-8mzhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:05.642594 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.642570    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-model-cache" pod "47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" (UID: "47fba953-aa96-4b41-9cc5-f00d6c6b2ffb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:05.642868 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.642848    4925 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tmp-dir\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:05.642925 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.642874    4925 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-home\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:05.642925 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.642887    4925 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-model-cache\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:05.642925 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.642904    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mzhn\" (UniqueName: \"kubernetes.io/projected/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-kube-api-access-8mzhn\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:05.647247 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.647219    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tls-certs" pod "47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" (UID: "47fba953-aa96-4b41-9cc5-f00d6c6b2ffb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:00:05.647368 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.647252    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-dshm" pod "47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" (UID: "47fba953-aa96-4b41-9cc5-f00d6c6b2ffb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:05.744770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.744738    4925 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-tls-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:05.744770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.744767    4925 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb-dshm\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:05.951630 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:05.946110    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:06.020724 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.020688    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6" event={"ID":"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc","Type":"ContainerStarted","Data":"dfbd27ec5e0925b1326f65f0c991db88427e6cb13825136681893023803774be"}
Oct 30 19:00:06.022173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.022148    4925 generic.go:356] "Generic (PLEG): container finished" podID="47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" containerID="8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6" exitCode=0
Oct 30 19:00:06.022281 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.022202    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"
Oct 30 19:00:06.022281 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.022243    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm" event={"ID":"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb","Type":"ContainerDied","Data":"8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6"}
Oct 30 19:00:06.022355 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.022279    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm" event={"ID":"47fba953-aa96-4b41-9cc5-f00d6c6b2ffb","Type":"ContainerDied","Data":"309ec77ea33b304db9e0f5af9ebb42bc4dbe459dbb9e2a3517638a4ca0978221"}
Oct 30 19:00:06.022355 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.022302    4925 scope.go:122] "RemoveContainer" containerID="8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6"
Oct 30 19:00:06.024140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.024081    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd" event={"ID":"6ba5df2e-f25f-4ae2-8612-92d72aa3a520","Type":"ContainerStarted","Data":"55c5f57a0a40b8a1822b2afd1ec5dbf94901029baa87abb0aea5a41db77defb4"}
Oct 30 19:00:06.032557 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.032542    4925 scope.go:122] "RemoveContainer" containerID="8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6"
Oct 30 19:00:06.032848 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:06.032826    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6\": container with ID starting with 8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6 not found: ID does not exist" containerID="8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6"
Oct 30 19:00:06.032938 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.032854    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6"} err="failed to get container status \"8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6\": rpc error: code = NotFound desc = could not find container \"8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6\": container with ID starting with 8423fa2cddea3c61929feebd85cc456dbfd2dc74cd7b6ac1571acf524e580bd6 not found: ID does not exist"
Oct 30 19:00:06.039317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.039279    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6" podStartSLOduration=2.132122197 podStartE2EDuration="6.039267435s" totalImagesPullingTime="3.907145238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:06.037757727 +0000 UTC m=+1427.754102806" watchObservedRunningTime="2026-10-30 19:00:06.039267435 +0000 UTC m=+1427.755612542"
Oct 30 19:00:06.053287 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.053265    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"]
Oct 30 19:00:06.056123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.056081    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-93574a/ipp-route-93574a-kserve-8474dbd557-xvhmm"]
Oct 30 19:00:06.068166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.068117    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd" podStartSLOduration=2.081378108 podStartE2EDuration="6.068083157s" totalImagesPullingTime="3.986705049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:06.067331452 +0000 UTC m=+1427.783676533" watchObservedRunningTime="2026-10-30 19:00:06.068083157 +0000 UTC m=+1427.784428235"
Oct 30 19:00:06.424504 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.424476    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" path="/var/lib/kubelet/pods/47fba953-aa96-4b41-9cc5-f00d6c6b2ffb/volumes"
Oct 30 19:00:06.776188 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:06.776051    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:07.032196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:07.032082    4925 generic.go:356] "Generic (PLEG): container finished" podID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" containerID="55c5f57a0a40b8a1822b2afd1ec5dbf94901029baa87abb0aea5a41db77defb4" exitCode=60
Oct 30 19:00:07.032196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:07.032160    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd" event={"ID":"6ba5df2e-f25f-4ae2-8612-92d72aa3a520","Type":"ContainerDied","Data":"55c5f57a0a40b8a1822b2afd1ec5dbf94901029baa87abb0aea5a41db77defb4"}
Oct 30 19:00:07.032492 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:07.032477    4925 scope.go:122] "RemoveContainer" containerID="55c5f57a0a40b8a1822b2afd1ec5dbf94901029baa87abb0aea5a41db77defb4"
Oct 30 19:00:07.034052 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:07.034012    4925 generic.go:356] "Generic (PLEG): container finished" podID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" containerID="dfbd27ec5e0925b1326f65f0c991db88427e6cb13825136681893023803774be" exitCode=60
Oct 30 19:00:07.034227 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:07.034082    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6" event={"ID":"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc","Type":"ContainerDied","Data":"dfbd27ec5e0925b1326f65f0c991db88427e6cb13825136681893023803774be"}
Oct 30 19:00:07.034497 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:07.034483    4925 scope.go:122] "RemoveContainer" containerID="dfbd27ec5e0925b1326f65f0c991db88427e6cb13825136681893023803774be"
Oct 30 19:00:07.468113 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:07.468059    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:07.662573 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:07.662528    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:07.822007 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:07.821974    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"]
Oct 30 19:00:07.822336 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:07.822298    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerName="istio-proxy" containerID="cri-o://d51ecb472ffb43ca243ada97a27c657ee3979e0b984bd73e262cd63585f12d49" gracePeriod=30
Oct 30 19:00:07.970182 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:07.969046    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:08.079785 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:08.079752    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6" event={"ID":"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc","Type":"ContainerStarted","Data":"2fc024902957dc01b288c2a749a10e35864a88a0a6e144caa31049561a9c1020"}
Oct 30 19:00:08.082143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:08.082106    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd" event={"ID":"6ba5df2e-f25f-4ae2-8612-92d72aa3a520","Type":"ContainerStarted","Data":"a8286aff9629e761546e2b4a876124206cb082f238e6a83b1054f4add9db367c"}
Oct 30 19:00:08.462119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:08.458112    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:08.998471 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:08.998433    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"]
Oct 30 19:00:09.026186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:09.026152    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"]
Oct 30 19:00:09.093118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:09.092244    4925 generic.go:356] "Generic (PLEG): container finished" podID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" containerID="a8286aff9629e761546e2b4a876124206cb082f238e6a83b1054f4add9db367c" exitCode=60
Oct 30 19:00:09.093118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:09.092566    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd" event={"ID":"6ba5df2e-f25f-4ae2-8612-92d72aa3a520","Type":"ContainerDied","Data":"a8286aff9629e761546e2b4a876124206cb082f238e6a83b1054f4add9db367c"}
Oct 30 19:00:09.093118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:09.092657    4925 scope.go:122] "RemoveContainer" containerID="55c5f57a0a40b8a1822b2afd1ec5dbf94901029baa87abb0aea5a41db77defb4"
Oct 30 19:00:09.093118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:09.092788    4925 scope.go:122] "RemoveContainer" containerID="a8286aff9629e761546e2b4a876124206cb082f238e6a83b1054f4add9db367c"
Oct 30 19:00:09.093466 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:09.093190    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd_odh-ai-gateway-infra(6ba5df2e-f25f-4ae2-8612-92d72aa3a520)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520"
Oct 30 19:00:09.096090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:09.095828    4925 generic.go:356] "Generic (PLEG): container finished" podID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" containerID="2fc024902957dc01b288c2a749a10e35864a88a0a6e144caa31049561a9c1020" exitCode=60
Oct 30 19:00:09.096090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:09.095878    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6" event={"ID":"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc","Type":"ContainerDied","Data":"2fc024902957dc01b288c2a749a10e35864a88a0a6e144caa31049561a9c1020"}
Oct 30 19:00:09.097701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:09.097676    4925 scope.go:122] "RemoveContainer" containerID="2fc024902957dc01b288c2a749a10e35864a88a0a6e144caa31049561a9c1020"
Oct 30 19:00:09.098030 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:09.098009    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6_odh-ai-gateway-infra(68ca6d6e-610a-463c-b383-f2c9c6f5a9cc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc"
Oct 30 19:00:09.113397 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:09.113375    4925 scope.go:122] "RemoveContainer" containerID="dfbd27ec5e0925b1326f65f0c991db88427e6cb13825136681893023803774be"
Oct 30 19:00:09.166137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:09.165759    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:09.354185 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:09.353676    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:09.615925 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:09.615837    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:10.092799 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:10.092173    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:10.130854 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:10.130828    4925 scope.go:122] "RemoveContainer" containerID="2fc024902957dc01b288c2a749a10e35864a88a0a6e144caa31049561a9c1020"
Oct 30 19:00:10.131229 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:10.131202    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6_odh-ai-gateway-infra(68ca6d6e-610a-463c-b383-f2c9c6f5a9cc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc"
Oct 30 19:00:10.132879 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:10.132857    4925 scope.go:122] "RemoveContainer" containerID="a8286aff9629e761546e2b4a876124206cb082f238e6a83b1054f4add9db367c"
Oct 30 19:00:10.133223 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:10.133201    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd_odh-ai-gateway-infra(6ba5df2e-f25f-4ae2-8612-92d72aa3a520)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520"
Oct 30 19:00:10.866531 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:10.866495    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:11.561381 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:11.555089    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:12.311775 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:12.311739    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:13.179035 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.176431    4925 generic.go:356] "Generic (PLEG): container finished" podID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerID="d51ecb472ffb43ca243ada97a27c657ee3979e0b984bd73e262cd63585f12d49" exitCode=0
Oct 30 19:00:13.179035 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.176610    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" event={"ID":"e41a16b6-cb27-412e-90a6-5374b69b8a76","Type":"ContainerDied","Data":"d51ecb472ffb43ca243ada97a27c657ee3979e0b984bd73e262cd63585f12d49"}
Oct 30 19:00:13.179035 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.176714    4925 scope.go:122] "RemoveContainer" containerID="5cc8988293ddfd81bf7c75ae9876ae7ee4971ae29a78bf43966d459bb3e49f10"
Oct 30 19:00:13.311220 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.310040    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:13.322481 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.322447    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 19:00:13.378239 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:13.377628    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:00:13.407770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.407722    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-socket\") pod \"e41a16b6-cb27-412e-90a6-5374b69b8a76\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") "
Oct 30 19:00:13.407938 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.407898    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e41a16b6-cb27-412e-90a6-5374b69b8a76-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41a16b6-cb27-412e-90a6-5374b69b8a76-istiod-ca-cert\") pod \"e41a16b6-cb27-412e-90a6-5374b69b8a76\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") "
Oct 30 19:00:13.408125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.408059    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-kube-api-access-vxhr5\" (UniqueName: \"kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-kube-api-access-vxhr5\") pod \"e41a16b6-cb27-412e-90a6-5374b69b8a76\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") "
Oct 30 19:00:13.408446 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.408393    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-data\") pod \"e41a16b6-cb27-412e-90a6-5374b69b8a76\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") "
Oct 30 19:00:13.408777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.408704    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-certs\") pod \"e41a16b6-cb27-412e-90a6-5374b69b8a76\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") "
Oct 30 19:00:13.409021 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.408953    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-podinfo\") pod \"e41a16b6-cb27-412e-90a6-5374b69b8a76\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") "
Oct 30 19:00:13.409195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.409141    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-envoy\") pod \"e41a16b6-cb27-412e-90a6-5374b69b8a76\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") "
Oct 30 19:00:13.409490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.409342    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-credential-socket\") pod \"e41a16b6-cb27-412e-90a6-5374b69b8a76\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") "
Oct 30 19:00:13.409490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.409437    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-token\" (UniqueName: \"kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-token\") pod \"e41a16b6-cb27-412e-90a6-5374b69b8a76\" (UID: \"e41a16b6-cb27-412e-90a6-5374b69b8a76\") "
Oct 30 19:00:13.409638 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.408389    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-socket" pod "e41a16b6-cb27-412e-90a6-5374b69b8a76" (UID: "e41a16b6-cb27-412e-90a6-5374b69b8a76"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:13.409638 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.408843    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e41a16b6-cb27-412e-90a6-5374b69b8a76-istiod-ca-cert" pod "e41a16b6-cb27-412e-90a6-5374b69b8a76" (UID: "e41a16b6-cb27-412e-90a6-5374b69b8a76"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:13.409638 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.409144    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-data" pod "e41a16b6-cb27-412e-90a6-5374b69b8a76" (UID: "e41a16b6-cb27-412e-90a6-5374b69b8a76"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:13.409638 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.409578    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-certs" pod "e41a16b6-cb27-412e-90a6-5374b69b8a76" (UID: "e41a16b6-cb27-412e-90a6-5374b69b8a76"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:13.410707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.410372    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41a16b6-cb27-412e-90a6-5374b69b8a76-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:13.410707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.410401    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:13.410707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.410416    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:13.410707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.410430    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:13.411522 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.411276    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-credential-socket" pod "e41a16b6-cb27-412e-90a6-5374b69b8a76" (UID: "e41a16b6-cb27-412e-90a6-5374b69b8a76"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:13.416873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.414544    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-token" pod "e41a16b6-cb27-412e-90a6-5374b69b8a76" (UID: "e41a16b6-cb27-412e-90a6-5374b69b8a76"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:13.416873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.416696    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-podinfo" pod "e41a16b6-cb27-412e-90a6-5374b69b8a76" (UID: "e41a16b6-cb27-412e-90a6-5374b69b8a76"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:13.417522 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.417497    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-envoy" pod "e41a16b6-cb27-412e-90a6-5374b69b8a76" (UID: "e41a16b6-cb27-412e-90a6-5374b69b8a76"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:13.419516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.419486    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-kube-api-access-vxhr5" pod "e41a16b6-cb27-412e-90a6-5374b69b8a76" (UID: "e41a16b6-cb27-412e-90a6-5374b69b8a76"). InnerVolumeSpecName "kube-api-access-vxhr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:13.511358 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.511326    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:13.511358 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.511356    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:13.511551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.511369    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41a16b6-cb27-412e-90a6-5374b69b8a76-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:13.511551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.511381    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:13.511551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:13.511393    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxhr5\" (UniqueName: \"kubernetes.io/projected/e41a16b6-cb27-412e-90a6-5374b69b8a76-kube-api-access-vxhr5\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:14.004234 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.004169    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv" podUID="8184da4d-3c5f-4628-af7d-f5e70b2029ad" containerName="payload-processing" containerID="cri-o://521378f8af3487a8708812f13ddada50b12ffd9850880a10c89398e551c6842f" gracePeriod=25
Oct 30 19:00:14.031120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.027459    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs" podUID="450d1376-252e-4a85-8b48-3337bce4e5d3" containerName="payload-pre-processing" containerID="cri-o://fe8d0181b99a61229d0b05c789ee48d297feddcb6ec8f3f7fdddd3568ef64cde" gracePeriod=25
Oct 30 19:00:14.081119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.080795    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:14.193032 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.192814    4925 generic.go:356] "Generic (PLEG): container finished" podID="450d1376-252e-4a85-8b48-3337bce4e5d3" containerID="fe8d0181b99a61229d0b05c789ee48d297feddcb6ec8f3f7fdddd3568ef64cde" exitCode=0
Oct 30 19:00:14.193032 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.192955    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs" event={"ID":"450d1376-252e-4a85-8b48-3337bce4e5d3","Type":"ContainerDied","Data":"fe8d0181b99a61229d0b05c789ee48d297feddcb6ec8f3f7fdddd3568ef64cde"}
Oct 30 19:00:14.199550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.198359    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22" event={"ID":"e41a16b6-cb27-412e-90a6-5374b69b8a76","Type":"ContainerDied","Data":"afebe8080be7d790512e78b43163c3d6990adcfc76c1b078e63658a54616976d"}
Oct 30 19:00:14.199550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.198421    4925 scope.go:122] "RemoveContainer" containerID="d51ecb472ffb43ca243ada97a27c657ee3979e0b984bd73e262cd63585f12d49"
Oct 30 19:00:14.199550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.198530    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"
Oct 30 19:00:14.212750 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.212580    4925 generic.go:356] "Generic (PLEG): container finished" podID="8184da4d-3c5f-4628-af7d-f5e70b2029ad" containerID="521378f8af3487a8708812f13ddada50b12ffd9850880a10c89398e551c6842f" exitCode=0
Oct 30 19:00:14.212750 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.212681    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv" event={"ID":"8184da4d-3c5f-4628-af7d-f5e70b2029ad","Type":"ContainerDied","Data":"521378f8af3487a8708812f13ddada50b12ffd9850880a10c89398e551c6842f"}
Oct 30 19:00:14.248854 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.248796    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"]
Oct 30 19:00:14.254689 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.254611    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-10d367-openshift-default-6fd7bbffc-8vt22"]
Oct 30 19:00:14.454928 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.454770    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" path="/var/lib/kubelet/pods/e41a16b6-cb27-412e-90a6-5374b69b8a76/volumes"
Oct 30 19:00:14.589761 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.589721    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"
Oct 30 19:00:14.606482 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.606456    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"
Oct 30 19:00:14.648862 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.648831    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8184da4d-3c5f-4628-af7d-f5e70b2029ad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8184da4d-3c5f-4628-af7d-f5e70b2029ad-plugins-config-volume\") pod \"8184da4d-3c5f-4628-af7d-f5e70b2029ad\" (UID: \"8184da4d-3c5f-4628-af7d-f5e70b2029ad\") "
Oct 30 19:00:14.649014 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.648936    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/450d1376-252e-4a85-8b48-3337bce4e5d3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/450d1376-252e-4a85-8b48-3337bce4e5d3-plugins-config-volume\") pod \"450d1376-252e-4a85-8b48-3337bce4e5d3\" (UID: \"450d1376-252e-4a85-8b48-3337bce4e5d3\") "
Oct 30 19:00:14.649114 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.649081    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/450d1376-252e-4a85-8b48-3337bce4e5d3-kube-api-access-vfm5r\" (UniqueName: \"kubernetes.io/projected/450d1376-252e-4a85-8b48-3337bce4e5d3-kube-api-access-vfm5r\") pod \"450d1376-252e-4a85-8b48-3337bce4e5d3\" (UID: \"450d1376-252e-4a85-8b48-3337bce4e5d3\") "
Oct 30 19:00:14.649233 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.649206    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8184da4d-3c5f-4628-af7d-f5e70b2029ad-kube-api-access-7lggs\" (UniqueName: \"kubernetes.io/projected/8184da4d-3c5f-4628-af7d-f5e70b2029ad-kube-api-access-7lggs\") pod \"8184da4d-3c5f-4628-af7d-f5e70b2029ad\" (UID: \"8184da4d-3c5f-4628-af7d-f5e70b2029ad\") "
Oct 30 19:00:14.649320 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.649271    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8184da4d-3c5f-4628-af7d-f5e70b2029ad-plugins-config-volume" pod "8184da4d-3c5f-4628-af7d-f5e70b2029ad" (UID: "8184da4d-3c5f-4628-af7d-f5e70b2029ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:14.649417 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.649391    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/450d1376-252e-4a85-8b48-3337bce4e5d3-plugins-config-volume" pod "450d1376-252e-4a85-8b48-3337bce4e5d3" (UID: "450d1376-252e-4a85-8b48-3337bce4e5d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:14.649745 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.649719    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8184da4d-3c5f-4628-af7d-f5e70b2029ad-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:14.649930 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.649749    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/450d1376-252e-4a85-8b48-3337bce4e5d3-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:14.651662 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.651631    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8184da4d-3c5f-4628-af7d-f5e70b2029ad-kube-api-access-7lggs" pod "8184da4d-3c5f-4628-af7d-f5e70b2029ad" (UID: "8184da4d-3c5f-4628-af7d-f5e70b2029ad"). InnerVolumeSpecName "kube-api-access-7lggs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:14.651885 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.651865    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/450d1376-252e-4a85-8b48-3337bce4e5d3-kube-api-access-vfm5r" pod "450d1376-252e-4a85-8b48-3337bce4e5d3" (UID: "450d1376-252e-4a85-8b48-3337bce4e5d3"). InnerVolumeSpecName "kube-api-access-vfm5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:14.751141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.751077    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lggs\" (UniqueName: \"kubernetes.io/projected/8184da4d-3c5f-4628-af7d-f5e70b2029ad-kube-api-access-7lggs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:14.751141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:14.751131    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfm5r\" (UniqueName: \"kubernetes.io/projected/450d1376-252e-4a85-8b48-3337bce4e5d3-kube-api-access-vfm5r\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:15.033021 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:15.032769    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:15.221182 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:15.221087    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"
Oct 30 19:00:15.221182 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:15.221136    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs" event={"ID":"450d1376-252e-4a85-8b48-3337bce4e5d3","Type":"ContainerDied","Data":"f465706ef3d7803f87b7e51e3d47c8a3172904f06193e545b2aa4042065f75a7"}
Oct 30 19:00:15.221690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:15.221211    4925 scope.go:122] "RemoveContainer" containerID="fe8d0181b99a61229d0b05c789ee48d297feddcb6ec8f3f7fdddd3568ef64cde"
Oct 30 19:00:15.224032 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:15.224006    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv" event={"ID":"8184da4d-3c5f-4628-af7d-f5e70b2029ad","Type":"ContainerDied","Data":"ed3259f3b15ec7490087100e6752d0b9ad03ad6f7f64e8d0889c34096f2853c4"}
Oct 30 19:00:15.224230 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:15.224016    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"
Oct 30 19:00:15.234163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:15.234136    4925 scope.go:122] "RemoveContainer" containerID="521378f8af3487a8708812f13ddada50b12ffd9850880a10c89398e551c6842f"
Oct 30 19:00:15.250185 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:15.249770    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"]
Oct 30 19:00:15.253652 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:15.253628    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-3d1aa8-785bb8bc95-6tmqs"]
Oct 30 19:00:15.263482 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:15.263453    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"]
Oct 30 19:00:15.267506 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:15.267484    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-3d1aa8-7596c58d6b-mjvxv"]
Oct 30 19:00:15.689123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:15.688075    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:16.121168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.117017    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"]
Oct 30 19:00:16.121168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.117292    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp" podUID="5bcc374b-bcc5-4ddd-ae99-22b8b53ae078" containerName="maas-api" containerID="cri-o://d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c" gracePeriod=30
Oct 30 19:00:16.237558 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.237522    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:16.427640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.427611    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:16.516700 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.507863    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="450d1376-252e-4a85-8b48-3337bce4e5d3" path="/var/lib/kubelet/pods/450d1376-252e-4a85-8b48-3337bce4e5d3/volumes"
Oct 30 19:00:16.516700 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.508696    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8184da4d-3c5f-4628-af7d-f5e70b2029ad" path="/var/lib/kubelet/pods/8184da4d-3c5f-4628-af7d-f5e70b2029ad/volumes"
Oct 30 19:00:16.628776 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.628716    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:16.725054 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.725017    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 19:00:16.823001 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.822968    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-maas-api-tls\") pod \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") "
Oct 30 19:00:16.823213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.823134    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-kube-api-access-jvjfr\" (UniqueName: \"kubernetes.io/projected/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-kube-api-access-jvjfr\") pod \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") "
Oct 30 19:00:16.823213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.823172    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:16.823339 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.823226    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-metrics-certs\") pod \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") "
Oct 30 19:00:16.823339 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.823313    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-openshift-service-ca\") pod \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\" (UID: \"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078\") "
Oct 30 19:00:16.823973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.823929    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-openshift-service-ca" pod "5bcc374b-bcc5-4ddd-ae99-22b8b53ae078" (UID: "5bcc374b-bcc5-4ddd-ae99-22b8b53ae078"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:16.826525 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.826496    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-maas-api-tls" pod "5bcc374b-bcc5-4ddd-ae99-22b8b53ae078" (UID: "5bcc374b-bcc5-4ddd-ae99-22b8b53ae078"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:00:16.826634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.826529    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-metrics-certs" pod "5bcc374b-bcc5-4ddd-ae99-22b8b53ae078" (UID: "5bcc374b-bcc5-4ddd-ae99-22b8b53ae078"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:00:16.826634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.826559    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-kube-api-access-jvjfr" pod "5bcc374b-bcc5-4ddd-ae99-22b8b53ae078" (UID: "5bcc374b-bcc5-4ddd-ae99-22b8b53ae078"). InnerVolumeSpecName "kube-api-access-jvjfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:16.924765 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.924731    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvjfr\" (UniqueName: \"kubernetes.io/projected/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-kube-api-access-jvjfr\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.924765 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.924762    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.924765 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.924773    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:16.925006 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:16.924781    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:17.037325 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:17.037244    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:17.244565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:17.244525    4925 generic.go:356] "Generic (PLEG): container finished" podID="5bcc374b-bcc5-4ddd-ae99-22b8b53ae078" containerID="d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c" exitCode=0
Oct 30 19:00:17.244993 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:17.244625    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"
Oct 30 19:00:17.244993 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:17.244637    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp" event={"ID":"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078","Type":"ContainerDied","Data":"d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c"}
Oct 30 19:00:17.244993 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:17.244727    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp" event={"ID":"5bcc374b-bcc5-4ddd-ae99-22b8b53ae078","Type":"ContainerDied","Data":"4c35d6b3514fe3748592f686e3d3113577be96c100224c026879b2b139b7d07a"}
Oct 30 19:00:17.244993 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:17.244752    4925 scope.go:122] "RemoveContainer" containerID="d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c"
Oct 30 19:00:17.257769 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:17.257749    4925 scope.go:122] "RemoveContainer" containerID="d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c"
Oct 30 19:00:17.258073 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:17.258053    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c\": container with ID starting with d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c not found: ID does not exist" containerID="d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c"
Oct 30 19:00:17.258212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:17.258080    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c"} err="failed to get container status \"d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c\": rpc error: code = NotFound desc = could not find container \"d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c\": container with ID starting with d88e02542098d9a44cc33c0df26199782edbfd5edfb1d0980584d251b3872e0c not found: ID does not exist"
Oct 30 19:00:17.270870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:17.270837    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"]
Oct 30 19:00:17.288152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:17.288077    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-3d1aa8-69db5c9c66-t28xp"]
Oct 30 19:00:17.553982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:17.553829    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:18.234122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:18.230673    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:18.424363 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:18.424331    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5bcc374b-bcc5-4ddd-ae99-22b8b53ae078" path="/var/lib/kubelet/pods/5bcc374b-bcc5-4ddd-ae99-22b8b53ae078/volumes"
Oct 30 19:00:19.059715 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:19.059679    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:19.414765 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:19.414739    4925 scope.go:122] "RemoveContainer" containerID="a8286aff9629e761546e2b4a876124206cb082f238e6a83b1054f4add9db367c"
Oct 30 19:00:19.702590 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:19.702490    4925 scope.go:122] "RemoveContainer" containerID="f512bf4c4b824dae4dbcb05bf53b1b434963fc983641804376f4705ead414314"
Oct 30 19:00:19.861120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:19.858598    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:20.292699 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:20.292655    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd" event={"ID":"6ba5df2e-f25f-4ae2-8612-92d72aa3a520","Type":"ContainerStarted","Data":"5da70781e356530b50641ad8e4c8b359649e9d355729ef7a58651014a2055fa8"}
Oct 30 19:00:20.622293 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:20.613831    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:21.302812 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:21.302779    4925 generic.go:356] "Generic (PLEG): container finished" podID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" containerID="5da70781e356530b50641ad8e4c8b359649e9d355729ef7a58651014a2055fa8" exitCode=60
Oct 30 19:00:21.303308 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:21.302956    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd" event={"ID":"6ba5df2e-f25f-4ae2-8612-92d72aa3a520","Type":"ContainerDied","Data":"5da70781e356530b50641ad8e4c8b359649e9d355729ef7a58651014a2055fa8"}
Oct 30 19:00:21.303308 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:21.302999    4925 scope.go:122] "RemoveContainer" containerID="a8286aff9629e761546e2b4a876124206cb082f238e6a83b1054f4add9db367c"
Oct 30 19:00:21.303461 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:21.303442    4925 scope.go:122] "RemoveContainer" containerID="5da70781e356530b50641ad8e4c8b359649e9d355729ef7a58651014a2055fa8"
Oct 30 19:00:21.303838 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:21.303804    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd_odh-ai-gateway-infra(6ba5df2e-f25f-4ae2-8612-92d72aa3a520)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520"
Oct 30 19:00:21.334298 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:21.334270    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"]
Oct 30 19:00:21.418121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:21.415285    4925 scope.go:122] "RemoveContainer" containerID="2fc024902957dc01b288c2a749a10e35864a88a0a6e144caa31049561a9c1020"
Oct 30 19:00:21.433134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:21.432839    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:22.348143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:22.347992    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:22.409119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:22.399677    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6" event={"ID":"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc","Type":"ContainerStarted","Data":"a30fb42d691e3ffcde88bf83183b1efa92fd117f03c5978b4fbd9bc9aba11fef"}
Oct 30 19:00:22.617745 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:22.617634    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"
Oct 30 19:00:22.662698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:22.661551    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-maas-api-service-ca\") pod \"6ba5df2e-f25f-4ae2-8612-92d72aa3a520\" (UID: \"6ba5df2e-f25f-4ae2-8612-92d72aa3a520\") "
Oct 30 19:00:22.662698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:22.661758    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-kube-api-access-qtfcb\" (UniqueName: \"kubernetes.io/projected/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-kube-api-access-qtfcb\") pod \"6ba5df2e-f25f-4ae2-8612-92d72aa3a520\" (UID: \"6ba5df2e-f25f-4ae2-8612-92d72aa3a520\") "
Oct 30 19:00:22.666945 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:22.666914    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-maas-api-service-ca" pod "6ba5df2e-f25f-4ae2-8612-92d72aa3a520" (UID: "6ba5df2e-f25f-4ae2-8612-92d72aa3a520"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:22.793680 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:22.793636    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-maas-api-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.799556 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:22.799504    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-kube-api-access-qtfcb" pod "6ba5df2e-f25f-4ae2-8612-92d72aa3a520" (UID: "6ba5df2e-f25f-4ae2-8612-92d72aa3a520"). InnerVolumeSpecName "kube-api-access-qtfcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.894934 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:22.894847    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtfcb\" (UniqueName: \"kubernetes.io/projected/6ba5df2e-f25f-4ae2-8612-92d72aa3a520-kube-api-access-qtfcb\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:23.197122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:23.196507    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:23.409754 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:23.409720    4925 generic.go:356] "Generic (PLEG): container finished" podID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" containerID="a30fb42d691e3ffcde88bf83183b1efa92fd117f03c5978b4fbd9bc9aba11fef" exitCode=60
Oct 30 19:00:23.410269 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:23.409788    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6" event={"ID":"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc","Type":"ContainerDied","Data":"a30fb42d691e3ffcde88bf83183b1efa92fd117f03c5978b4fbd9bc9aba11fef"}
Oct 30 19:00:23.410269 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:23.409851    4925 scope.go:122] "RemoveContainer" containerID="2fc024902957dc01b288c2a749a10e35864a88a0a6e144caa31049561a9c1020"
Oct 30 19:00:23.410269 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:23.410212    4925 scope.go:122] "RemoveContainer" containerID="a30fb42d691e3ffcde88bf83183b1efa92fd117f03c5978b4fbd9bc9aba11fef"
Oct 30 19:00:23.410529 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:23.410509    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6_odh-ai-gateway-infra(68ca6d6e-610a-463c-b383-f2c9c6f5a9cc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc"
Oct 30 19:00:23.411643 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:23.411621    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd" event={"ID":"6ba5df2e-f25f-4ae2-8612-92d72aa3a520","Type":"ContainerDied","Data":"044da68751a97e6d58c733c66806456c0f831670cee5cd35ea22f7ca996072c0"}
Oct 30 19:00:23.411768 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:23.411679    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"
Oct 30 19:00:23.425932 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:23.425912    4925 scope.go:122] "RemoveContainer" containerID="5da70781e356530b50641ad8e4c8b359649e9d355729ef7a58651014a2055fa8"
Oct 30 19:00:23.443259 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:23.443239    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"]
Oct 30 19:00:23.445907 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:23.445889    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"]
Oct 30 19:00:23.454330 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:23.454280    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-a-6a327a-29889780-lnqmd"]
Oct 30 19:00:24.006117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:24.004999    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:24.214385 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:24.214343    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:24.427343 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:24.427316    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" path="/var/lib/kubelet/pods/6ba5df2e-f25f-4ae2-8612-92d72aa3a520/volumes"
Oct 30 19:00:24.645122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:24.644812    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:24.813792 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:24.813548    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"
Oct 30 19:00:24.835364 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:24.835333    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:24.924410 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:24.924380    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-kube-api-access-79d6w\" (UniqueName: \"kubernetes.io/projected/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-kube-api-access-79d6w\") pod \"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc\" (UID: \"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc\") "
Oct 30 19:00:24.924573 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:24.924480    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-maas-api-service-ca\") pod \"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc\" (UID: \"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc\") "
Oct 30 19:00:24.924824 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:24.924798    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-maas-api-service-ca" pod "68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" (UID: "68ca6d6e-610a-463c-b383-f2c9c6f5a9cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:24.924980 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:24.924967    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-maas-api-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:24.926706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:24.926684    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-kube-api-access-79d6w" pod "68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" (UID: "68ca6d6e-610a-463c-b383-f2c9c6f5a9cc"). InnerVolumeSpecName "kube-api-access-79d6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:25.026033 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:25.026005    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79d6w\" (UniqueName: \"kubernetes.io/projected/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc-kube-api-access-79d6w\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:25.431411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:25.431378    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6" event={"ID":"68ca6d6e-610a-463c-b383-f2c9c6f5a9cc","Type":"ContainerDied","Data":"b680460df7fb5a463d6b65c07e0d30e9e07a8a4563f7f71a111217d18cc7e18a"}
Oct 30 19:00:25.431411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:25.431409    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"
Oct 30 19:00:25.431917 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:25.431430    4925 scope.go:122] "RemoveContainer" containerID="a30fb42d691e3ffcde88bf83183b1efa92fd117f03c5978b4fbd9bc9aba11fef"
Oct 30 19:00:25.453543 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:25.453507    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"]
Oct 30 19:00:25.460689 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:25.460662    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-praxis-b-a014d1-29889780-dvdc6"]
Oct 30 19:00:25.588821 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:25.588607    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:26.340167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:26.336386    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:26.423284 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:26.423250    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" path="/var/lib/kubelet/pods/68ca6d6e-610a-463c-b383-f2c9c6f5a9cc/volumes"
Oct 30 19:00:26.919874 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:26.919835    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:27.215186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:27.214926    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:27.881119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:27.879619    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:28.205626 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:28.205584    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:28.583555 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:28.583458    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:29.392526 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:29.387049    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:30.048061 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:30.048028    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:30.767132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:30.767076    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:31.481031 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:31.480973    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:32.310121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:32.305221    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:32.984127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:32.977686    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:33.957250 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:33.957066    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:34.624266 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:34.624233    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:35.303169 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:35.298595    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:35.623082 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:35.622996    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:35.850425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:35.850387    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"]
Oct 30 19:00:35.850752 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:35.850696    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82" podUID="9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" containerName="istio-proxy" containerID="cri-o://ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f" gracePeriod=30
Oct 30 19:00:35.940859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:35.940812    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:36.143531 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:36.143489    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:36.510377 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:36.510343    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:37.505132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:37.504720    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:38.897171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:38.878366    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:39.870210 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:39.870172    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:40.721446 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:40.721404    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:41.139687 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.139663    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 19:00:41.234408 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.234374    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-data\") pod \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") "
Oct 30 19:00:41.234408 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.234411    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-envoy\") pod \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") "
Oct 30 19:00:41.234646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.234454    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-socket\") pod \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") "
Oct 30 19:00:41.234646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.234500    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-credential-socket\") pod \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") "
Oct 30 19:00:41.234646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.234541    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-podinfo\") pod \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") "
Oct 30 19:00:41.234646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.234631    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istiod-ca-cert\") pod \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") "
Oct 30 19:00:41.234870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.234687    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-token\" (UniqueName: \"kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-token\") pod \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") "
Oct 30 19:00:41.234870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.234727    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-data" pod "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" (UID: "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:41.234870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.234758    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-socket" pod "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" (UID: "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:41.234870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.234792    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-kube-api-access-69bfg\" (UniqueName: \"kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-kube-api-access-69bfg\") pod \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") "
Oct 30 19:00:41.234870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.234834    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-certs\") pod \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\" (UID: \"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b\") "
Oct 30 19:00:41.235166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.234852    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-credential-socket" pod "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" (UID: "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:41.235166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.235063    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istiod-ca-cert" pod "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" (UID: "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:41.235283 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.235210    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-certs" pod "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" (UID: "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:41.235460 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.235440    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:41.235569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.235464    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:41.235569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.235478    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:41.235569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.235491    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:41.235569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.235505    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:41.237143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.237113    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-token" pod "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" (UID: "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:41.237540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.237520    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-envoy" pod "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" (UID: "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:41.237730 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.237710    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-podinfo" pod "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" (UID: "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:41.237794 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.237746    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-kube-api-access-69bfg" pod "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" (UID: "9fb5bb8c-bb43-4b3b-84de-1f1793672d4b"). InnerVolumeSpecName "kube-api-access-69bfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:41.336777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.336688    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69bfg\" (UniqueName: \"kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-kube-api-access-69bfg\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:41.336777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.336715    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:41.336777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.336726    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:41.336777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.336734    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:41.684044 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.682962    4925 generic.go:356] "Generic (PLEG): container finished" podID="9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" containerID="ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f" exitCode=0
Oct 30 19:00:41.684341 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.684307    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82" event={"ID":"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b","Type":"ContainerDied","Data":"ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f"}
Oct 30 19:00:41.684506 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.684377    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82" event={"ID":"9fb5bb8c-bb43-4b3b-84de-1f1793672d4b","Type":"ContainerDied","Data":"7f83518d9f890a90f97ba1bc0fdd9c687c177eb92cae863df5dd8568167e0510"}
Oct 30 19:00:41.684506 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.684406    4925 scope.go:122] "RemoveContainer" containerID="ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f"
Oct 30 19:00:41.684626 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.684566    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"
Oct 30 19:00:41.700344 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.700318    4925 scope.go:122] "RemoveContainer" containerID="ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f"
Oct 30 19:00:41.700999 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:41.700954    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f\": container with ID starting with ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f not found: ID does not exist" containerID="ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f"
Oct 30 19:00:41.701124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.700998    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f"} err="failed to get container status \"ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f\": rpc error: code = NotFound desc = could not find container \"ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f\": container with ID starting with ce59cf758ad4fdecf0e836ce98bf9e7bc7324f58910cf9b7058805ec8fc9935f not found: ID does not exist"
Oct 30 19:00:41.716517 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.716495    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"]
Oct 30 19:00:41.720258 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.720237    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-93574a-openshift-default-54946f844f-cgr82"]
Oct 30 19:00:41.955930 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:41.955847    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:42.433710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:42.433665    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" path="/var/lib/kubelet/pods/9fb5bb8c-bb43-4b3b-84de-1f1793672d4b/volumes"
Oct 30 19:00:42.768207 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:42.768122    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:43.306420 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:00:43.304082    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:00:43.679949 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:43.679477    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:44.276166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276128    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"]
Oct 30 19:00:44.276912 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276883    4925 cpu_manager.go:414] "containerMap: removing container" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerName="istio-proxy"
Oct 30 19:00:44.276912 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276908    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerName="istio-proxy"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276921    4925 cpu_manager.go:414] "containerMap: removing container" podUID="5bcc374b-bcc5-4ddd-ae99-22b8b53ae078" containerName="maas-api"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276926    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bcc374b-bcc5-4ddd-ae99-22b8b53ae078" containerName="maas-api"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276934    4925 cpu_manager.go:414] "containerMap: removing container" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" containerName="cleanup"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276939    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" containerName="cleanup"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276958    4925 cpu_manager.go:414] "containerMap: removing container" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276963    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276969    4925 cpu_manager.go:414] "containerMap: removing container" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" containerName="cleanup"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276973    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" containerName="cleanup"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276981    4925 cpu_manager.go:414] "containerMap: removing container" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" containerName="cleanup"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.276985    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" containerName="cleanup"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277004    4925 cpu_manager.go:414] "containerMap: removing container" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" containerName="cleanup"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277011    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" containerName="cleanup"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277023    4925 cpu_manager.go:414] "containerMap: removing container" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277030    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277041    4925 cpu_manager.go:414] "containerMap: removing container" podUID="8184da4d-3c5f-4628-af7d-f5e70b2029ad" containerName="payload-processing"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277047    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8184da4d-3c5f-4628-af7d-f5e70b2029ad" containerName="payload-processing"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277064    4925 cpu_manager.go:414] "containerMap: removing container" podUID="9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" containerName="istio-proxy"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277072    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" containerName="istio-proxy"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277080    4925 cpu_manager.go:414] "containerMap: removing container" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" containerName="cleanup"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277085    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" containerName="cleanup"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277114    4925 cpu_manager.go:414] "containerMap: removing container" podUID="450d1376-252e-4a85-8b48-3337bce4e5d3" containerName="payload-pre-processing"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277119    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="450d1376-252e-4a85-8b48-3337bce4e5d3" containerName="payload-pre-processing"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277131    4925 cpu_manager.go:414] "containerMap: removing container" podUID="47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" containerName="main"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277136    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" containerName="main"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277143    4925 cpu_manager.go:414] "containerMap: removing container" podUID="03a6f6b5-669d-449a-86b4-bbf8d0329f58" containerName="maas-api"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277148    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03a6f6b5-669d-449a-86b4-bbf8d0329f58" containerName="maas-api"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277157    4925 cpu_manager.go:414] "containerMap: removing container" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" containerName="cleanup"
Oct 30 19:00:44.277166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277164    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" containerName="cleanup"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277307    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerName="istio-proxy"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277323    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="5bcc374b-bcc5-4ddd-ae99-22b8b53ae078" containerName="maas-api"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277335    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" containerName="cleanup"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277349    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277357    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" containerName="cleanup"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277367    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ba5df2e-f25f-4ae2-8612-92d72aa3a520" containerName="cleanup"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277378    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" containerName="cleanup"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277386    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="d699d780-921f-48ae-8ce4-27fdd806b52d" containerName="istio-proxy"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277393    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="8184da4d-3c5f-4628-af7d-f5e70b2029ad" containerName="payload-processing"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277402    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fb5bb8c-bb43-4b3b-84de-1f1793672d4b" containerName="istio-proxy"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277416    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="450d1376-252e-4a85-8b48-3337bce4e5d3" containerName="payload-pre-processing"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277426    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="47fba953-aa96-4b41-9cc5-f00d6c6b2ffb" containerName="main"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277434    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="03a6f6b5-669d-449a-86b4-bbf8d0329f58" containerName="maas-api"
Oct 30 19:00:44.279263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.277443    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" containerName="cleanup"
Oct 30 19:00:44.282378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.282357    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.333597 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.333571    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"]
Oct 30 19:00:44.334289 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.334265    4925 cpu_manager.go:414] "containerMap: removing container" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerName="istio-proxy"
Oct 30 19:00:44.334289 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.334287    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerName="istio-proxy"
Oct 30 19:00:44.334470 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.334399    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="68ca6d6e-610a-463c-b383-f2c9c6f5a9cc" containerName="cleanup"
Oct 30 19:00:44.334470 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.334411    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="e41a16b6-cb27-412e-90a6-5374b69b8a76" containerName="istio-proxy"
Oct 30 19:00:44.338163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.337952    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.341226 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.341205    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"]
Oct 30 19:00:44.390524 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.390489    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"]
Oct 30 19:00:44.393718 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.393692    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-data\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.393844 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.393729    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.393844 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.393758    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-envoy\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.393934 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.393878    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-credential-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.393934 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.393923    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvgb6\" (UniqueName: \"kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-kube-api-access-nvgb6\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.394008 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.393990    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-podinfo\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.394040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.394025    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istiod-ca-cert\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.394071 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.394055    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-certs\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.394118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.394081    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istiod-ca-cert\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.394166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.394154    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-envoy\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.394213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.394199    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-podinfo\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.394271 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.394258    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m74vf\" (UniqueName: \"kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-kube-api-access-m74vf\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.394308 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.394291    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-certs\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.394343 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.394321    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-token\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.394408 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.394394    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-token\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.394494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.394480    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-credential-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.394612 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.394599    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.394661 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.394647    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-data\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.496239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496204    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-token\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.496402 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496264    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-credential-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.496402 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496321    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.496402 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496347    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-data\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.496580 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496485    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-data\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.496580 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496542    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.496580 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496573    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-envoy\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.496743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496610    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-credential-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.496743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496655    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvgb6\" (UniqueName: \"kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-kube-api-access-nvgb6\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.496743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496720    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-podinfo\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.496901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496746    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istiod-ca-cert\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.496901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496775    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-certs\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.496901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496783    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.496901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496800    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istiod-ca-cert\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.496901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496806    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-data\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.496901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496811    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-data\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.496901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496849    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-envoy\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.496901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496896    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-podinfo\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.497462 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496926    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.497462 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496949    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m74vf\" (UniqueName: \"kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-kube-api-access-m74vf\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.497462 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.496981    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-certs\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.497462 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.497008    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-token\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.497462 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.497267    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-credential-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.497735 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.497508    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istiod-ca-cert\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.497735 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.497617    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-credential-socket\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.498432 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.497855    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-certs\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.498432 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.498238    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-certs\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.498432 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.498377    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istiod-ca-cert\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.499455 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.499435    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-envoy\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.499580 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.499563    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-envoy\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.500173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.500153    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-podinfo\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.500433 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.500404    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-podinfo\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.509274 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.509229    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-token\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.509384 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.509299    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m74vf\" (UniqueName: \"kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-kube-api-access-m74vf\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.509554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.509533    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-token\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.509807 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.509783    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvgb6\" (UniqueName: \"kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-kube-api-access-nvgb6\") pod \"e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") " pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.596914 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.596840    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:44.679908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.679652    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:44.917417 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.913401    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"]
Oct 30 19:00:44.967121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.966455    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:44.977861 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.977822    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:44.977955 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.977914    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:44.978021 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:44.977960    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:45.018061 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:45.018032    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"]
Oct 30 19:00:45.019281 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:00:45.019256    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36905cc_0aa9_4b0a_aca4_02ba5efae873.slice/crio-a50fbc46176a101dbd815a4c5b6939eeb054aa911931147d0da685893fe2e0ff WatchSource:0}: Error finding container a50fbc46176a101dbd815a4c5b6939eeb054aa911931147d0da685893fe2e0ff: Status 404 returned error can't find the container with id a50fbc46176a101dbd815a4c5b6939eeb054aa911931147d0da685893fe2e0ff
Oct 30 19:00:45.021956 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:45.021924    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:45.022028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:45.021994    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:45.022073 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:45.022025    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:45.735628 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:45.735586    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps" event={"ID":"93bd9c54-8bda-4c70-962a-fe0f73e80a7a","Type":"ContainerStarted","Data":"695500f3785244e3e738f0029b8dda857ca6dd971ad313c18b28d23da2326abe"}
Oct 30 19:00:45.736129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:45.735664    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps" event={"ID":"93bd9c54-8bda-4c70-962a-fe0f73e80a7a","Type":"ContainerStarted","Data":"a98f9c3ff2ba8cf42d3fd572ac75b9d8dda7f4603a7d292119559568e6452434"}
Oct 30 19:00:45.737847 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:45.737819    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp" event={"ID":"b36905cc-0aa9-4b0a-aca4-02ba5efae873","Type":"ContainerStarted","Data":"639a884f5ef7e03265f77b062348de14c4dc58f39eb6fe4c1335d7790ca01077"}
Oct 30 19:00:45.737978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:45.737869    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp" event={"ID":"b36905cc-0aa9-4b0a-aca4-02ba5efae873","Type":"ContainerStarted","Data":"a50fbc46176a101dbd815a4c5b6939eeb054aa911931147d0da685893fe2e0ff"}
Oct 30 19:00:45.763645 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:45.763592    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps" podStartSLOduration=1.7635630629999999 podStartE2EDuration="1.763563063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:45.756135491 +0000 UTC m=+1467.472480571" watchObservedRunningTime="2026-10-30 19:00:45.763563063 +0000 UTC m=+1467.479908161"
Oct 30 19:00:45.790983 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:45.790828    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp" podStartSLOduration=1.7908092 podStartE2EDuration="1.7908092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:45.7808352 +0000 UTC m=+1467.497180277" watchObservedRunningTime="2026-10-30 19:00:45.7908092 +0000 UTC m=+1467.507154281"
Oct 30 19:00:46.236240 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:46.226638    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:46.606173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:46.605163    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:46.606484 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:46.606360    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.158:15021/healthz/ready\": dial tcp 10.132.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:00:46.606484 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:46.606433    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps" podUID="93bd9c54-8bda-4c70-962a-fe0f73e80a7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.158:15021/healthz/ready\": dial tcp 10.132.0.158:15021: connect: connection refused"
Oct 30 19:00:46.689202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:46.689160    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:46.698272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:46.698243    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.159:15021/healthz/ready\": dial tcp 10.132.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:00:46.698618 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:46.698496    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp" podUID="b36905cc-0aa9-4b0a-aca4-02ba5efae873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.159:15021/healthz/ready\": dial tcp 10.132.0.159:15021: connect: connection refused"
Oct 30 19:00:47.523121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:47.515004    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:47.606122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:47.601521    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.158:15021/healthz/ready\": dial tcp 10.132.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:00:47.606122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:47.601586    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps" podUID="93bd9c54-8bda-4c70-962a-fe0f73e80a7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.158:15021/healthz/ready\": dial tcp 10.132.0.158:15021: connect: connection refused"
Oct 30 19:00:47.683120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:47.682451    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.159:15021/healthz/ready\": dial tcp 10.132.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:00:47.683120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:47.682513    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp" podUID="b36905cc-0aa9-4b0a-aca4-02ba5efae873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.159:15021/healthz/ready\": dial tcp 10.132.0.159:15021: connect: connection refused"
Oct 30 19:00:48.605122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:48.604536    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.158:15021/healthz/ready\": dial tcp 10.132.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:00:48.605122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:48.604599    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps" podUID="93bd9c54-8bda-4c70-962a-fe0f73e80a7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.158:15021/healthz/ready\": dial tcp 10.132.0.158:15021: connect: connection refused"
Oct 30 19:00:48.650122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:48.649090    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:48.694125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:48.693247    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.159:15021/healthz/ready\": dial tcp 10.132.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:00:48.694125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:48.693332    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp" podUID="b36905cc-0aa9-4b0a-aca4-02ba5efae873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.159:15021/healthz/ready\": dial tcp 10.132.0.159:15021: connect: connection refused"
Oct 30 19:00:48.912886 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:48.912852    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:49.599145 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:49.589824    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:49.605118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:49.599455    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.158:15021/healthz/ready\": dial tcp 10.132.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:00:49.605118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:49.599511    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps" podUID="93bd9c54-8bda-4c70-962a-fe0f73e80a7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.158:15021/healthz/ready\": dial tcp 10.132.0.158:15021: connect: connection refused"
Oct 30 19:00:49.688121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:49.680498    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.159:15021/healthz/ready\": dial tcp 10.132.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:00:49.688121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:49.680558    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp" podUID="b36905cc-0aa9-4b0a-aca4-02ba5efae873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.159:15021/healthz/ready\": dial tcp 10.132.0.159:15021: connect: connection refused"
Oct 30 19:00:50.506123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:50.505882    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:50.603130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:50.602612    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.158:15021/healthz/ready\": dial tcp 10.132.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:00:50.603130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:50.602688    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps" podUID="93bd9c54-8bda-4c70-962a-fe0f73e80a7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.158:15021/healthz/ready\": dial tcp 10.132.0.158:15021: connect: connection refused"
Oct 30 19:00:50.682547 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:50.682511    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.159:15021/healthz/ready\": dial tcp 10.132.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:00:50.689226 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:50.689191    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp" podUID="b36905cc-0aa9-4b0a-aca4-02ba5efae873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.159:15021/healthz/ready\": dial tcp 10.132.0.159:15021: connect: connection refused"
Oct 30 19:00:51.364772 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:51.361283    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:51.597530 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:51.597485    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.158:15021/healthz/ready\": dial tcp 10.132.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:00:51.597738 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:51.597586    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps" podUID="93bd9c54-8bda-4c70-962a-fe0f73e80a7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.158:15021/healthz/ready\": dial tcp 10.132.0.158:15021: connect: connection refused"
Oct 30 19:00:51.681158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:51.681120    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.159:15021/healthz/ready\": dial tcp 10.132.0.159:15021: connect: connection refused" start-of-body=
Oct 30 19:00:51.681360 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:51.681190    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp" podUID="b36905cc-0aa9-4b0a-aca4-02ba5efae873" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.159:15021/healthz/ready\": dial tcp 10.132.0.159:15021: connect: connection refused"
Oct 30 19:00:52.066817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:52.066728    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:52.903634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:52.903514    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:52.939440 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:52.938457    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:52.939851 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:52.939772    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:52.940634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:52.940608    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:00:52.946118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:52.944423    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:52.946118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:52.945544    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:52.948392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:52.948373    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:00:53.295764 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.295731    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"]
Oct 30 19:00:53.296626 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.296596    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s" podUID="ea6dbfed-a952-4c82-be12-9d477b34978c" containerName="maas-api" containerID="cri-o://b0fdda1d8b5ad93666f982b527a5c468106d036360a011c193859a5bc3da7b18" gracePeriod=30
Oct 30 19:00:53.353945 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.353917    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"]
Oct 30 19:00:53.354201 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.354174    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6" podUID="c7eea034-caf9-4b41-8cbb-77715993d9d1" containerName="payload-processing" containerID="cri-o://3dc7ec9857008f76254a92e460508995b32ba6a0cd7b5c82a01e6bc27464c750" gracePeriod=30
Oct 30 19:00:53.377551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.377520    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"]
Oct 30 19:00:53.384124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.377762    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8" podUID="b234d7d5-b327-43a5-ac64-aa2a377ae3a6" containerName="payload-pre-processing" containerID="cri-o://b7b7cda22a317f25ec4fcb5ab8c5fedb91771004f78c3e33e8e793d869a4f332" gracePeriod=30
Oct 30 19:00:53.425668 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.425637    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:53.461363 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.461331    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"]
Oct 30 19:00:53.468219 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.468194    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.503272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.494729    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"]
Oct 30 19:00:53.505432 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.505396    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"]
Oct 30 19:00:53.512019 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.509640    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.517386 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.515642    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"]
Oct 30 19:00:53.517386 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.515685    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"]
Oct 30 19:00:53.517386 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.515768    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.586062 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.557686    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"]
Oct 30 19:00:53.634877 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.608909    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.634877 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.608965    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.634877 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.609011    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.634877 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.634680    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643315    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbzdw\" (UniqueName: \"kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-kube-api-access-dbzdw\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643420    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf20214-a424-49a8-b45a-02236a44a19f-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643484    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643510    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643539    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4s5g\" (UniqueName: \"kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-kube-api-access-d4s5g\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643567    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643595    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643632    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643676    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643709    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf20214-a424-49a8-b45a-02236a44a19f-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643740    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643805    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643836    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbgcp\" (UniqueName: \"kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-kube-api-access-mbgcp\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643875    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643905    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643942    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c005ba-9d2d-4d5d-aeac-2e762809092c-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.643977    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.644017    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.644054    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.644117    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.644143    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.644205    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.644240    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.644813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.644287    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.749125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.748832    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.749125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.748885    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.749125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.748918    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4s5g\" (UniqueName: \"kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-kube-api-access-d4s5g\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.749125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.748944    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.749125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.748973    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.749125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.749008    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.749125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.749047    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.749125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.749077    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf20214-a424-49a8-b45a-02236a44a19f-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.749125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.749125    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756277    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756327    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbgcp\" (UniqueName: \"kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-kube-api-access-mbgcp\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756368    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756392    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756426    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c005ba-9d2d-4d5d-aeac-2e762809092c-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756460    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756501    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756534    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756575    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756599    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756667    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756703    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756744    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756787    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756813    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756867    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.756982    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbzdw\" (UniqueName: \"kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-kube-api-access-dbzdw\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.757069    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf20214-a424-49a8-b45a-02236a44a19f-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.757970    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf20214-a424-49a8-b45a-02236a44a19f-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.758449    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.758784    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.759428    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.759668    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-certs\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.759927    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.782894    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s" podUID="ea6dbfed-a952-4c82-be12-9d477b34978c" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.120:8443/health\": dial tcp 10.132.0.120:8443: connect: connection refused"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.783692    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.794231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.789359    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.823758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.807752    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.823758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.809448    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-data\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.823758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.809681    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.823758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.809889    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.823758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.811623    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-credential-socket\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.823758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.812224    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c005ba-9d2d-4d5d-aeac-2e762809092c-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.823758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.812820    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istiod-ca-cert\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.823758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.819550    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.823758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.819883    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.827848 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.827564    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbgcp\" (UniqueName: \"kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-kube-api-access-mbgcp\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.833468 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.833401    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:53.835880 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.835851    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.838468 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.838291    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.839054 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.839002    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.839726 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.839667    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.840458 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.840286    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf20214-a424-49a8-b45a-02236a44a19f-istio-podinfo\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.841120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.841051    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-envoy\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.841285 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.841254    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4s5g\" (UniqueName: \"kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-kube-api-access-d4s5g\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.841779 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.841753    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-istio-token\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.843868 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.843844    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbzdw\" (UniqueName: \"kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-kube-api-access-dbzdw\") pod \"e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") " pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:53.854604 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.854582    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:53.922949 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.922920    4925 generic.go:356] "Generic (PLEG): container finished" podID="ea6dbfed-a952-4c82-be12-9d477b34978c" containerID="b0fdda1d8b5ad93666f982b527a5c468106d036360a011c193859a5bc3da7b18" exitCode=0
Oct 30 19:00:53.923296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.923048    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s" event={"ID":"ea6dbfed-a952-4c82-be12-9d477b34978c","Type":"ContainerDied","Data":"b0fdda1d8b5ad93666f982b527a5c468106d036360a011c193859a5bc3da7b18"}
Oct 30 19:00:53.925124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.925072    4925 generic.go:356] "Generic (PLEG): container finished" podID="b234d7d5-b327-43a5-ac64-aa2a377ae3a6" containerID="b7b7cda22a317f25ec4fcb5ab8c5fedb91771004f78c3e33e8e793d869a4f332" exitCode=0
Oct 30 19:00:53.925270 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.925252    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8" event={"ID":"b234d7d5-b327-43a5-ac64-aa2a377ae3a6","Type":"ContainerDied","Data":"b7b7cda22a317f25ec4fcb5ab8c5fedb91771004f78c3e33e8e793d869a4f332"}
Oct 30 19:00:53.928012 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.927932    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:53.928229 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.928208    4925 generic.go:356] "Generic (PLEG): container finished" podID="c7eea034-caf9-4b41-8cbb-77715993d9d1" containerID="3dc7ec9857008f76254a92e460508995b32ba6a0cd7b5c82a01e6bc27464c750" exitCode=0
Oct 30 19:00:53.928993 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:53.928967    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6" event={"ID":"c7eea034-caf9-4b41-8cbb-77715993d9d1","Type":"ContainerDied","Data":"3dc7ec9857008f76254a92e460508995b32ba6a0cd7b5c82a01e6bc27464c750"}
Oct 30 19:00:54.024121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.019112    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:54.076760 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.076340    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"
Oct 30 19:00:54.099995 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.099958    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 19:00:54.102311 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.102237    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"
Oct 30 19:00:54.119918 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.119236    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:54.171420 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.169260    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7eea034-caf9-4b41-8cbb-77715993d9d1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7eea034-caf9-4b41-8cbb-77715993d9d1-plugins-config-volume\") pod \"c7eea034-caf9-4b41-8cbb-77715993d9d1\" (UID: \"c7eea034-caf9-4b41-8cbb-77715993d9d1\") "
Oct 30 19:00:54.171420 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.169379    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7eea034-caf9-4b41-8cbb-77715993d9d1-kube-api-access-bfw4c\" (UniqueName: \"kubernetes.io/projected/c7eea034-caf9-4b41-8cbb-77715993d9d1-kube-api-access-bfw4c\") pod \"c7eea034-caf9-4b41-8cbb-77715993d9d1\" (UID: \"c7eea034-caf9-4b41-8cbb-77715993d9d1\") "
Oct 30 19:00:54.182666 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.182611    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:54.182780 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.182756    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:54.182869 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.182846    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:54.198866 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.198836    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7eea034-caf9-4b41-8cbb-77715993d9d1-plugins-config-volume" pod "c7eea034-caf9-4b41-8cbb-77715993d9d1" (UID: "c7eea034-caf9-4b41-8cbb-77715993d9d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:54.213588 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.213165    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7eea034-caf9-4b41-8cbb-77715993d9d1-kube-api-access-bfw4c" pod "c7eea034-caf9-4b41-8cbb-77715993d9d1" (UID: "c7eea034-caf9-4b41-8cbb-77715993d9d1"). InnerVolumeSpecName "kube-api-access-bfw4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:54.216225 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.216081    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"]
Oct 30 19:00:54.216620 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.216485    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:54.277231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.274598    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ea6dbfed-a952-4c82-be12-9d477b34978c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6dbfed-a952-4c82-be12-9d477b34978c-openshift-service-ca\") pod \"ea6dbfed-a952-4c82-be12-9d477b34978c\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") "
Oct 30 19:00:54.277231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.274691    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-maas-api-tls\") pod \"ea6dbfed-a952-4c82-be12-9d477b34978c\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") "
Oct 30 19:00:54.277231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.274725    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-plugins-config-volume\") pod \"b234d7d5-b327-43a5-ac64-aa2a377ae3a6\" (UID: \"b234d7d5-b327-43a5-ac64-aa2a377ae3a6\") "
Oct 30 19:00:54.277231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.274862    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea6dbfed-a952-4c82-be12-9d477b34978c-kube-api-access-9zdwg\" (UniqueName: \"kubernetes.io/projected/ea6dbfed-a952-4c82-be12-9d477b34978c-kube-api-access-9zdwg\") pod \"ea6dbfed-a952-4c82-be12-9d477b34978c\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") "
Oct 30 19:00:54.277231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.275033    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-kube-api-access-dmg9q\" (UniqueName: \"kubernetes.io/projected/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-kube-api-access-dmg9q\") pod \"b234d7d5-b327-43a5-ac64-aa2a377ae3a6\" (UID: \"b234d7d5-b327-43a5-ac64-aa2a377ae3a6\") "
Oct 30 19:00:54.277231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.275066    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-metrics-certs\") pod \"ea6dbfed-a952-4c82-be12-9d477b34978c\" (UID: \"ea6dbfed-a952-4c82-be12-9d477b34978c\") "
Oct 30 19:00:54.277231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.275689    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfw4c\" (UniqueName: \"kubernetes.io/projected/c7eea034-caf9-4b41-8cbb-77715993d9d1-kube-api-access-bfw4c\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:54.277231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.275713    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7eea034-caf9-4b41-8cbb-77715993d9d1-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:54.277231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.276075    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-plugins-config-volume" pod "b234d7d5-b327-43a5-ac64-aa2a377ae3a6" (UID: "b234d7d5-b327-43a5-ac64-aa2a377ae3a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:54.277231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.276399    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea6dbfed-a952-4c82-be12-9d477b34978c-openshift-service-ca" pod "ea6dbfed-a952-4c82-be12-9d477b34978c" (UID: "ea6dbfed-a952-4c82-be12-9d477b34978c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:54.281399 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.278561    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-metrics-certs" pod "ea6dbfed-a952-4c82-be12-9d477b34978c" (UID: "ea6dbfed-a952-4c82-be12-9d477b34978c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:00:54.281399 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.278852    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-maas-api-tls" pod "ea6dbfed-a952-4c82-be12-9d477b34978c" (UID: "ea6dbfed-a952-4c82-be12-9d477b34978c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:00:54.281399 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.281071    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-kube-api-access-dmg9q" pod "b234d7d5-b327-43a5-ac64-aa2a377ae3a6" (UID: "b234d7d5-b327-43a5-ac64-aa2a377ae3a6"). InnerVolumeSpecName "kube-api-access-dmg9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:54.281399 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.281281    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea6dbfed-a952-4c82-be12-9d477b34978c-kube-api-access-9zdwg" pod "ea6dbfed-a952-4c82-be12-9d477b34978c" (UID: "ea6dbfed-a952-4c82-be12-9d477b34978c"). InnerVolumeSpecName "kube-api-access-9zdwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:54.367724 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.367700    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"]
Oct 30 19:00:54.377332 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.377210    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea6dbfed-a952-4c82-be12-9d477b34978c-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:54.377332 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.377248    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:54.377332 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.377262    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:54.377332 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.377279    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zdwg\" (UniqueName: \"kubernetes.io/projected/ea6dbfed-a952-4c82-be12-9d477b34978c-kube-api-access-9zdwg\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:54.377332 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.377294    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmg9q\" (UniqueName: \"kubernetes.io/projected/b234d7d5-b327-43a5-ac64-aa2a377ae3a6-kube-api-access-dmg9q\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:54.377332 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.377308    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea6dbfed-a952-4c82-be12-9d477b34978c-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:54.382255 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:00:54.382225    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb8bc37_2bab_4541_a4e0_e3bc9fb47d73.slice/crio-814cfa65a290bca433d42a30b52309e1ee76172f53ff5b817332f23e31fb83ae WatchSource:0}: Error finding container 814cfa65a290bca433d42a30b52309e1ee76172f53ff5b817332f23e31fb83ae: Status 404 returned error can't find the container with id 814cfa65a290bca433d42a30b52309e1ee76172f53ff5b817332f23e31fb83ae
Oct 30 19:00:54.388358 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.386302    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:54.388358 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.386386    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:54.388358 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.386425    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:54.469251 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.469221    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:54.493587 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.493550    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"]
Oct 30 19:00:54.509701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.506970    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:54.509701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.507057    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:54.509701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.507119    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:54.947859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.947817    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" event={"ID":"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73","Type":"ContainerStarted","Data":"14dffd84df729d6f05a07c47f828692bfbcf3b7e62002646d744cc98c79678ed"}
Oct 30 19:00:54.948165 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.948141    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" event={"ID":"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73","Type":"ContainerStarted","Data":"814cfa65a290bca433d42a30b52309e1ee76172f53ff5b817332f23e31fb83ae"}
Oct 30 19:00:54.952564 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.952524    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s" event={"ID":"ea6dbfed-a952-4c82-be12-9d477b34978c","Type":"ContainerDied","Data":"e3d44c8d58885605ab3d56c5f353281c567104079dcbeb10f1af18d5f94b095e"}
Oct 30 19:00:54.952705 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.952585    4925 scope.go:122] "RemoveContainer" containerID="b0fdda1d8b5ad93666f982b527a5c468106d036360a011c193859a5bc3da7b18"
Oct 30 19:00:54.952705 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.952534    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"
Oct 30 19:00:54.954851 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.954816    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8" event={"ID":"b234d7d5-b327-43a5-ac64-aa2a377ae3a6","Type":"ContainerDied","Data":"bd67922efd6495ec64738b8bc5f77af122c3db9028375b18dfab3a48f6b5f04b"}
Oct 30 19:00:54.954953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.954938    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"
Oct 30 19:00:54.983357 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.965274    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" event={"ID":"86c005ba-9d2d-4d5d-aeac-2e762809092c","Type":"ContainerStarted","Data":"f6ca2dbc120b4727fa87611a6d1cb758d76ad796ff65404298c2c83705da832d"}
Oct 30 19:00:54.983357 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.965334    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" event={"ID":"86c005ba-9d2d-4d5d-aeac-2e762809092c","Type":"ContainerStarted","Data":"accbe5d85bbe844478b680c45fbea5a4229a1ae302adff4b3a6f2aec39b78def"}
Oct 30 19:00:54.983357 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.967978    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6" event={"ID":"c7eea034-caf9-4b41-8cbb-77715993d9d1","Type":"ContainerDied","Data":"e365114b56c6635ecbb22af39db9e0e9f09467878b7de4f30b170b676258747a"}
Oct 30 19:00:54.983357 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.968121    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"
Oct 30 19:00:54.983357 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.973219    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" event={"ID":"0bf20214-a424-49a8-b45a-02236a44a19f","Type":"ContainerStarted","Data":"3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c"}
Oct 30 19:00:54.983357 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.973263    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" event={"ID":"0bf20214-a424-49a8-b45a-02236a44a19f","Type":"ContainerStarted","Data":"1f9dc71a75c6208c9b5fc5946f5958174998712a25573a4eb23675e6c0d9eb4c"}
Oct 30 19:00:55.000119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:54.999814    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podStartSLOduration=1.999794657 podStartE2EDuration="1.999794657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:54.981584266 +0000 UTC m=+1476.697929355" watchObservedRunningTime="2026-10-30 19:00:54.999794657 +0000 UTC m=+1476.716139736"
Oct 30 19:00:55.011042 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.011019    4925 scope.go:122] "RemoveContainer" containerID="b7b7cda22a317f25ec4fcb5ab8c5fedb91771004f78c3e33e8e793d869a4f332"
Oct 30 19:00:55.016612 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.016590    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"]
Oct 30 19:00:55.024115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.021162    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-24fceb-657cc9b99f-4tcw8"]
Oct 30 19:00:55.026275 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.026261    4925 scope.go:122] "RemoveContainer" containerID="3dc7ec9857008f76254a92e460508995b32ba6a0cd7b5c82a01e6bc27464c750"
Oct 30 19:00:55.035389 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.035341    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podStartSLOduration=2.035326999 podStartE2EDuration="2.035326999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:55.031489804 +0000 UTC m=+1476.747834878" watchObservedRunningTime="2026-10-30 19:00:55.035326999 +0000 UTC m=+1476.751672077"
Oct 30 19:00:55.055137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.055079    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podStartSLOduration=2.055063089 podStartE2EDuration="2.055063089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:55.052767676 +0000 UTC m=+1476.769112754" watchObservedRunningTime="2026-10-30 19:00:55.055063089 +0000 UTC m=+1476.771408166"
Oct 30 19:00:55.098148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.096662    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"]
Oct 30 19:00:55.100809 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.100786    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-24fceb-7f8c8b9948-p6m4s"]
Oct 30 19:00:55.114950 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.114925    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"]
Oct 30 19:00:55.121019 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.120994    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:00:55.121167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.121039    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-24fceb-79f8dc5dd-d5lx6"]
Oct 30 19:00:55.122315 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.122280    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:00:55.122408 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.122334    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:00:55.223036 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.222890    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:55.855383 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.855347    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:00:55.860272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.860225    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:00:55.860568 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.860540    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:00:55.945132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.944171    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:00:55.951448 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.950236    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:00:55.951448 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:55.950307    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:00:56.124397 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:56.124305    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:00:56.124665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:56.124621    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:00:56.523121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:56.522248    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b234d7d5-b327-43a5-ac64-aa2a377ae3a6" path="/var/lib/kubelet/pods/b234d7d5-b327-43a5-ac64-aa2a377ae3a6/volumes"
Oct 30 19:00:56.523121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:56.522872    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7eea034-caf9-4b41-8cbb-77715993d9d1" path="/var/lib/kubelet/pods/c7eea034-caf9-4b41-8cbb-77715993d9d1/volumes"
Oct 30 19:00:56.523529 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:56.523505    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea6dbfed-a952-4c82-be12-9d477b34978c" path="/var/lib/kubelet/pods/ea6dbfed-a952-4c82-be12-9d477b34978c/volumes"
Oct 30 19:00:56.576500 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:56.576456    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:56.925178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:56.905921    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:00:56.925178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:56.905985    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:00:56.933129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:56.930192    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:00:56.933129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:56.930247    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:00:57.040194 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:57.040148    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"]
Oct 30 19:00:57.052271 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:57.052237    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"]
Oct 30 19:00:57.157254 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:57.157218    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:00:57.157455 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:57.157289    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:00:57.247065 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:57.245449    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:57.485129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:57.474051    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:57.820123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:57.813251    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:57.872124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:57.858517    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:00:57.872124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:57.858581    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:00:57.933730 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:57.933692    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:00:57.934790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:57.934756    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:00:58.126204 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.124468    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:00:58.126204 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.124542    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:00:58.180193 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.180164    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"]
Oct 30 19:00:58.190152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.186414    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:58.400192 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.397379    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:58.638178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.607187    4925 cpu_manager.go:414] "containerMap: removing container" podUID="c7eea034-caf9-4b41-8cbb-77715993d9d1" containerName="payload-processing"
Oct 30 19:00:58.638178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.607222    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7eea034-caf9-4b41-8cbb-77715993d9d1" containerName="payload-processing"
Oct 30 19:00:58.638178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.607282    4925 cpu_manager.go:414] "containerMap: removing container" podUID="b234d7d5-b327-43a5-ac64-aa2a377ae3a6" containerName="payload-pre-processing"
Oct 30 19:00:58.638178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.607291    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b234d7d5-b327-43a5-ac64-aa2a377ae3a6" containerName="payload-pre-processing"
Oct 30 19:00:58.638178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.607315    4925 cpu_manager.go:414] "containerMap: removing container" podUID="ea6dbfed-a952-4c82-be12-9d477b34978c" containerName="maas-api"
Oct 30 19:00:58.638178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.607322    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea6dbfed-a952-4c82-be12-9d477b34978c" containerName="maas-api"
Oct 30 19:00:58.638178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.607566    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="b234d7d5-b327-43a5-ac64-aa2a377ae3a6" containerName="payload-pre-processing"
Oct 30 19:00:58.638178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.607590    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea6dbfed-a952-4c82-be12-9d477b34978c" containerName="maas-api"
Oct 30 19:00:58.638178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.607608    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7eea034-caf9-4b41-8cbb-77715993d9d1" containerName="payload-processing"
Oct 30 19:00:58.655208 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.643226    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.700120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.687615    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c08b4a-e175-4674-abc4-b783e98090d2-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.700120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.687693    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.700120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.687741    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.700120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.687767    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpb6t\" (UniqueName: \"kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-kube-api-access-dpb6t\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.700120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.687801    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c08b4a-e175-4674-abc4-b783e98090d2-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.700120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.687868    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.700120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.687918    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.700120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.687963    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.700120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.688117    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.753142 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.751848    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"]
Oct 30 19:00:58.754136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.753064    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:58.770344 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.770318    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"]
Oct 30 19:00:58.807145 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.806473    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:58.828189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.810459    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.828189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.810521    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.828189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.810552    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpb6t\" (UniqueName: \"kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-kube-api-access-dpb6t\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.828189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.810587    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c08b4a-e175-4674-abc4-b783e98090d2-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.828189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.810650    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.828189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.810685    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.828189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.810728    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.828189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.810850    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.828189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.810983    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c08b4a-e175-4674-abc4-b783e98090d2-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.828189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.820741    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c08b4a-e175-4674-abc4-b783e98090d2-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.828189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.827078    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.838392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.834687    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.838392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.834928    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.843186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.843145    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.843979 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.843956    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c08b4a-e175-4674-abc4-b783e98090d2-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.861984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.861489    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:00:58.861984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.861540    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:00:58.865126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.863083    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"]
Oct 30 19:00:58.865126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.863156    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"]
Oct 30 19:00:58.865126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.863292    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:58.888184 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.879737    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.890786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.890752    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.894729 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.892973    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpb6t\" (UniqueName: \"kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-kube-api-access-dpb6t\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916124    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33f803d9-d817-4b7a-97df-24609789f8e6-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916193    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916236    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a19f8f-b107-4093-ac58-115127210713-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916304    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916340    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916368    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a19f8f-b107-4093-ac58-115127210713-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916425    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916461    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916532    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33f803d9-d817-4b7a-97df-24609789f8e6-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916606    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916637    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916686    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8d4z\" (UniqueName: \"kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-kube-api-access-v8d4z\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916744    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms97t\" (UniqueName: \"kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-kube-api-access-ms97t\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916782    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916829    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916861    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916889    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:58.918132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.916980    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:58.932024 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.931999    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:00:58.932306 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.932255    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:00:58.988210 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:58.988158    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"]
Oct 30 19:00:59.020964 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.020917    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a19f8f-b107-4093-ac58-115127210713-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.021290 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.021266    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.021439 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.021424    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.021560 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.021546    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a19f8f-b107-4093-ac58-115127210713-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.021694 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.021679    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.021812 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.021795    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.021930 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.021917    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33f803d9-d817-4b7a-97df-24609789f8e6-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.022063 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.022050    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.022196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.022179    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.022299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.022285    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8d4z\" (UniqueName: \"kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-kube-api-access-v8d4z\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.022437 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.022423    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms97t\" (UniqueName: \"kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-kube-api-access-ms97t\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.022554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.022539    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.022658 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.022646    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.022763 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.022750    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.038875 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.038837    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.039195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.039156    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.040232 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.040214    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33f803d9-d817-4b7a-97df-24609789f8e6-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.041223 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.041202    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.041478 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.038161    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.041612 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.038393    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.041727 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.039721    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.041845 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.022878    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a19f8f-b107-4093-ac58-115127210713-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.041951 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.040004    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-credential-socket\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.042052 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.023143    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.056208 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.023669    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.056363 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.049249    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33f803d9-d817-4b7a-97df-24609789f8e6-istiod-ca-cert\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.056510 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.037439    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-certs\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.056656 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.055405    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:00:59.056853 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.055729    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-data\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.076859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.076795    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33f803d9-d817-4b7a-97df-24609789f8e6-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.086666 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.086638    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a19f8f-b107-4093-ac58-115127210713-istio-podinfo\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.087572 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.087508    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8d4z\" (UniqueName: \"kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-kube-api-access-v8d4z\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.088165 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.088142    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.089868 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.089821    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-istio-token\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.108152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.108126    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.123924 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.123458    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:00:59.123924 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.123517    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:00:59.144248 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.133785    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms97t\" (UniqueName: \"kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-kube-api-access-ms97t\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.144248 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.134964    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-envoy\") pod \"e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") " pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.234642 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.233958    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:00:59.274685 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.273372    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:00:59.864246 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.860142    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:00:59.864246 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.860201    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:00:59.936504 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.934626    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:00:59.936504 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.934690    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:00:59.977826 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:00:59.977714    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:00.387905 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:00.381778    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:01:00.387905 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:00.381860    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:01:00.659201 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:00.659136    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:00.659678 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:00.659647    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:00.659996 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:00.659876    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:00.879120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:00.868818    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 19:01:00.919167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:00.905843    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:01:00.919167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:00.905926    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:01:00.978142 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:00.977272    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:01:00.978142 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:00.977355    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:01:01.123271 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.123236    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:01:01.123541 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.123512    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:01:01.323888 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.323788    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:01.357191 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:01:01.356295    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f803d9_d817_4b7a_97df_24609789f8e6.slice/crio-ca9cdbcf639661b0e4ac0ef1ce093a3bd435886dd0ad484ffb5cd7306dfc84eb WatchSource:0}: Error finding container ca9cdbcf639661b0e4ac0ef1ce093a3bd435886dd0ad484ffb5cd7306dfc84eb: Status 404 returned error can't find the container with id ca9cdbcf639661b0e4ac0ef1ce093a3bd435886dd0ad484ffb5cd7306dfc84eb
Oct 30 19:01:01.360741 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.360698    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:01.361056 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.361031    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:01.382631 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.382244    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:01.393973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.393918    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:01.587744 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.587641    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:01.686437 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.680355    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:01.686813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.686782    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:01.687570 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.682538    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:01.688584 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.675850    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:01:01.752619 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.752117    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"]
Oct 30 19:01:01.752619 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.752193    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"]
Oct 30 19:01:01.763369 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.761482    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" event={"ID":"52c08b4a-e175-4674-abc4-b783e98090d2","Type":"ContainerStarted","Data":"e48be01d2c1fcf49b838245859c7341217a54fca313bb520c87c84429999cabc"}
Oct 30 19:01:01.763369 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.761611    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"]
Oct 30 19:01:01.866829 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.866734    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:01:01.867393 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.867253    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:01:01.911811 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.896967    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:01.957155 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.956993    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:01:01.973371 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.958378    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:01:01.973371 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:01.958426    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:01:02.044763 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.044164    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w" podUID="8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6" containerName="payload-processing" containerID="cri-o://1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb" gracePeriod=25
Oct 30 19:01:02.054704 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.054161    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t" podUID="3823fc1b-a60f-4315-ae97-db1bc2e6974b" containerName="payload-pre-processing" containerID="cri-o://5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e" gracePeriod=25
Oct 30 19:01:02.124083 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.121513    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:01:02.124083 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.121565    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:01:02.753445 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.753399    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w" podUID="8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:01:02.836139 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.836084    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" event={"ID":"33f803d9-d817-4b7a-97df-24609789f8e6","Type":"ContainerStarted","Data":"fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32"}
Oct 30 19:01:02.836378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.836361    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" event={"ID":"33f803d9-d817-4b7a-97df-24609789f8e6","Type":"ContainerStarted","Data":"ca9cdbcf639661b0e4ac0ef1ce093a3bd435886dd0ad484ffb5cd7306dfc84eb"}
Oct 30 19:01:02.857968 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.857934    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:01:02.914135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.905185    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:01:02.914135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.880134    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podStartSLOduration=4.880114413 podStartE2EDuration="4.880114413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:02.878549897 +0000 UTC m=+1484.594894973" watchObservedRunningTime="2026-10-30 19:01:02.880114413 +0000 UTC m=+1484.596459490"
Oct 30 19:01:02.914135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.861559    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" event={"ID":"52c08b4a-e175-4674-abc4-b783e98090d2","Type":"ContainerStarted","Data":"93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be"}
Oct 30 19:01:02.924009 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.923953    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podStartSLOduration=4.923934843 podStartE2EDuration="4.923934843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:02.915047319 +0000 UTC m=+1484.631392398" watchObservedRunningTime="2026-10-30 19:01:02.923934843 +0000 UTC m=+1484.640279920"
Oct 30 19:01:03.071382 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:02.933627    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:01:03.071382 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.026753    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:01:03.071382 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.047710    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:01:03.071382 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.048052    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:03.071382 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.048337    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 19:01:03.071382 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.049737    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" event={"ID":"98a19f8f-b107-4093-ac58-115127210713","Type":"ContainerStarted","Data":"d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92"}
Oct 30 19:01:03.071382 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.049787    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" event={"ID":"98a19f8f-b107-4093-ac58-115127210713","Type":"ContainerStarted","Data":"f017d4d73923e012a2487b4c02e6cb6f64f8beba27d2312bda26d0f77f2f91e0"}
Oct 30 19:01:03.159152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.137258    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:01:03.159152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.138284    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:01:03.159152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.138332    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:01:03.190329 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.161001    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:03.190329 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.161058    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:03.190329 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.183383    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:03.228206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.227436    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:03.257994 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.251338    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:01:03.288902 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.258351    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:03.288902 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.258407    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:03.339456 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.339369    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:03.339699 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.339667    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:03.339962 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.339941    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:01:03.432394 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.432336    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podStartSLOduration=5.432313961 podStartE2EDuration="5.432313961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:03.360468394 +0000 UTC m=+1485.076813493" watchObservedRunningTime="2026-10-30 19:01:03.432313961 +0000 UTC m=+1485.148659038"
Oct 30 19:01:03.842153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.822416    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t" podUID="3823fc1b-a60f-4315-ae97-db1bc2e6974b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:01:03.842153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.822781    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:03.906328 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.906272    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:01:03.906658 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:03.906609    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:01:04.019578 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.019539    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:01:04.023197 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.023159    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:01:04.098879 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.097395    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:04.123490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.099154    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 19:01:04.123490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.104251    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:01:04.166154 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.135941    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:01:04.166154 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.135998    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:01:04.219188 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.197309    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:04.219188 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.197377    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:04.219188 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.203391    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:04.219188 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.204249    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 19:01:04.293319 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.267157    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:04.293319 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.268320    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 19:01:04.293319 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.290371    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:04.293319 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.290422    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:04.319143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.294221    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:04.370721 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.344110    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:04.454237 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.454191    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:04.635894 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.635804    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:04.875130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.871130    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:01:04.890990 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.890783    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:01:04.891862 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.891830    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:04.952156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:04.916407    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 19:01:05.146042 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.066435    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:05.146042 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.066492    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:05.156996 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.148908    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:01:05.156996 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.148966    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:01:05.156996 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.149047    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:01:05.156996 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.149067    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:01:05.156996 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.149640    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:05.156996 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.150201    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:05.227222 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.227180    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:01:05.276355 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.276312    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:05.404991 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.313191    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:05.405301 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.405265    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:05.434200 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.314370    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:05.434451 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.434418    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:05.434581 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.427365    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": context deadline exceeded"
Oct 30 19:01:05.933125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.893691    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:05.961122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.952460    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:01:05.967234 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:05.967188    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:01:06.016317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.016225    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:01:06.036862 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.036812    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:01:06.094600 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.094561    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:06.094908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.094877    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:06.133127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.131858    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:01:06.133127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.131919    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:01:06.167124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.156876    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:06.167124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.157203    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 19:01:06.238682 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.238568    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:06.238943 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.238907    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:06.266377 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.266322    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:06.275476 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.275441    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:06.275721 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.275691    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:06.325562 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.319270    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:06.848124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.845425    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:01:06.865195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.865144    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:01:06.866497 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.866454    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:01:06.907316 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:06.907265    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:07.022137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.014156    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:01:07.022137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.014233    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:01:07.068149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.064285    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:07.068149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.064345    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:07.193135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.191680    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:07.226122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.198801    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:01:07.226122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.198857    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:01:07.255888 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.255852    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:07.256194 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.256160    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:07.268232 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.268192    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:07.343116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.315796    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:07.443120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.442849    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:07.454170 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.321821    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:07.932174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.923848    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:07.937128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.936305    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:01:07.937128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:07.936818    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": dial tcp 10.132.0.162:15021: connect: connection refused"
Oct 30 19:01:08.017123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.013293    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:08.067583 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.067547    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:08.068192 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.068152    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:08.125420 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.125387    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:01:08.125668 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.125644    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": dial tcp 10.132.0.160:15021: connect: connection refused"
Oct 30 19:01:08.209175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.203923    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:08.213189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.213154    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 19:01:08.248916 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.248882    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:08.249242 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.249201    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:08.271284 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.270181    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:01:08.282739 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.282711    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:08.282993 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.282967    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:08.468172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.457396    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:08.729296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.729188    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:08.871227 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.871189    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:01:08.917123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.916184    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:08.917123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.916251    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:08.947397 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.945197    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:01:08.947847 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:08.947814    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:09.058698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.058198    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:09.058698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.058260    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:09.123442 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.123324    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:01:09.188131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.187512    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:09.215223 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.215178    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:09.215982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.215955    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:01:09.389202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.372219    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:01:09.389202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.373062    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:09.389202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.373128    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:09.411194 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.392212    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:09.428159 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.426983    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:09.428159 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.427038    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:09.512189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.502171    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:09.959319 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.959271    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 19:01:09.968257 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.968210    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:09.968483 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.968458    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:10.012120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:09.973175    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.162:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:10.012120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.001774    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.162:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:10.078128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.070532    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:10.228375 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.228258    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:10.228922 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.223889    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:10.229162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.085076    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:01:10.231118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.206357    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:10.231309 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.231289    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:10.270175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.249789    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:10.270175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.251312    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:10.270175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.251366    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:10.272585 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.272549    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:01:10.303111 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.297433    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:10.359119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.358579    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:10.384139 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.380805    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:10.609917 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.434153    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:01:10.609917 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.503447    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:10.621273 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.619585    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:10.781124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.774167    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": context deadline exceeded"
Oct 30 19:01:10.961128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:10.960840    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:11.046255 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.046216    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:11.046506 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.046477    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:11.060747 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.060703    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:11.060962 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.060937    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:11.243181 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.242477    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:11.243181 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.242541    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:11.262130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.258677    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:11.274257 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.274224    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:11.274519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.274494    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:11.279199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.279175    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:11.279499 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.279475    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:11.381233 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.381184    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:01:11.484179 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.481359    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:11.519400 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.519299    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:01:11.670470 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.670439    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:01:11.969144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:11.965249    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:12.071252 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.071201    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:12.071493 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.071465    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:12.098186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.098149    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:12.098437 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.098411    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:12.145119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.141856    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 19:01:12.236176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.235466    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:12.236176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.235525    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:12.296241 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.296202    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:12.317132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.312187    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:12.338219 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.330245    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:12.370755 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.347194    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:12.458124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.390251    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:01:12.544172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.519472    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:12.665884 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.665848    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:12.752333 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.752301    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:01:12.753565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.753438    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w" podUID="8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:01:12.765507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.765482    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:01:12.956083 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.934609    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 19:01:12.971251 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:12.971208    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:13.116318 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.116284    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:13.116599 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.116572    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:13.220855 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.212427    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 19:01:13.257210 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.257176    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:13.257475 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.257444    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:13.382132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.380922    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 19:01:13.454179 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.442509    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:13.454179 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.442584    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:13.492178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.478600    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 19:01:13.562386 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.562344    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:01:13.595596 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.578392    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:13.595851 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.595820    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.160:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:13.778177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.738551    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:01:13.921148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.918433    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:01:13.922066 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:13.922019    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t" podUID="3823fc1b-a60f-4315-ae97-db1bc2e6974b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:01:14.077147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:14.077030    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:01:14.105282 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:14.105245    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:14.194123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:14.158242    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:14.210343 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:01:14.210300    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:01:14.239313 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:14.239281    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:01:14.287123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:14.274477    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:14.287123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:14.274554    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:14.299120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:14.297556    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:01:14.303118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:14.301066    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:14.303118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:14.301199    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:14.752119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:14.748570    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:15.056073 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:15.055984    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:15.056291 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:15.056090    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:15.235255 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:15.235212    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused" start-of-body=
Oct 30 19:01:15.235495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:15.235287    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": dial tcp 10.132.0.164:15021: connect: connection refused"
Oct 30 19:01:15.274738 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:15.274704    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:01:15.274994 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:15.274966    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": dial tcp 10.132.0.165:15021: connect: connection refused"
Oct 30 19:01:16.031990 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:16.031143    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:16.078082 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:16.061467    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:01:16.078082 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:16.076551    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:01:16.407440 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:16.407357    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:01:16.410062 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:16.410040    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:01:17.256129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:17.251177    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:17.256129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:17.251249    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:17.530201 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:17.527267    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:18.074294 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:18.074226    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:18.074751 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:18.074697    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:18.212122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:18.199193    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:18.274555 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:18.274512    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:18.279212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:18.279182    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:18.279419 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:18.279395    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:18.518363 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:18.518319    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:18.595234 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:18.595190    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:18.963243 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:18.963189    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:19.130539 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.129197    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:19.130539 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.129272    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:19.160258 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.160214    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:19.205130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.204474    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:19.205356 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.205230    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 19:01:19.293174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.290445    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:19.293174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.291246    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:01:19.411147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.404281    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:19.411147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.404414    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:19.520545 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.520475    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:19.831078 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.648525    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:19.839855 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.839830    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 19:01:20.305424 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.904641    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 19:01:20.306725 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:19.964235    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:20.333645 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:20.333620    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 19:01:20.334481 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:20.189444    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/echo-simulator-kserve-64b6f8f96c-qht75" podUID="bd42a605-1c88-4212-9199-1f3572a08984" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:20.362388 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:20.223206    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:01:20.363015 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:20.271630    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:20.363243 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:20.363208    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:20.364088 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:20.304957    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:20.658180 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:20.646908    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.164:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:20.658477 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:20.658441    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.164:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:20.708147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:20.704173    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": context deadline exceeded"
Oct 30 19:01:20.875123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:20.863568    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:20.895124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:20.885246    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:20.968990 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:20.962619    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 19:01:21.147590 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:21.147546    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:01:21.248422 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:21.200287    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:01:21.378135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:21.377029    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:21.420240 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:21.420202    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:21.420519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:21.420490    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:21.435494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:21.435460    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:21.746504 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:21.746478    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 19:01:21.823989 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:21.823959    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 19:01:21.894981 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:21.887743    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:21.962551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:21.962502    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:21.996918 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:21.996891    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 19:01:22.027129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.026372    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:01:22.047824 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.046333    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:01:22.079295 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.074177    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 19:01:22.079295 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.074250    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:01:22.139018 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.138847    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 19:01:22.140373 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.140350    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:22.452909 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.452868    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"
Oct 30 19:01:22.506980 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.506952    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3823fc1b-a60f-4315-ae97-db1bc2e6974b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3823fc1b-a60f-4315-ae97-db1bc2e6974b-plugins-config-volume\") pod \"3823fc1b-a60f-4315-ae97-db1bc2e6974b\" (UID: \"3823fc1b-a60f-4315-ae97-db1bc2e6974b\") "
Oct 30 19:01:22.507736 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.507442    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3823fc1b-a60f-4315-ae97-db1bc2e6974b-plugins-config-volume" pod "3823fc1b-a60f-4315-ae97-db1bc2e6974b" (UID: "3823fc1b-a60f-4315-ae97-db1bc2e6974b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:22.508239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.507972    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3823fc1b-a60f-4315-ae97-db1bc2e6974b-kube-api-access-b92c2\" (UniqueName: \"kubernetes.io/projected/3823fc1b-a60f-4315-ae97-db1bc2e6974b-kube-api-access-b92c2\") pod \"3823fc1b-a60f-4315-ae97-db1bc2e6974b\" (UID: \"3823fc1b-a60f-4315-ae97-db1bc2e6974b\") "
Oct 30 19:01:22.508652 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.508619    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3823fc1b-a60f-4315-ae97-db1bc2e6974b-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:22.511259 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.511235    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3823fc1b-a60f-4315-ae97-db1bc2e6974b-kube-api-access-b92c2" pod "3823fc1b-a60f-4315-ae97-db1bc2e6974b" (UID: "3823fc1b-a60f-4315-ae97-db1bc2e6974b"). InnerVolumeSpecName "kube-api-access-b92c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:22.532575 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.532551    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"
Oct 30 19:01:22.609134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.609070    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-kube-api-access-f4rn5\" (UniqueName: \"kubernetes.io/projected/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-kube-api-access-f4rn5\") pod \"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6\" (UID: \"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6\") "
Oct 30 19:01:22.609296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.609207    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-plugins-config-volume\") pod \"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6\" (UID: \"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6\") "
Oct 30 19:01:22.609747 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.609714    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b92c2\" (UniqueName: \"kubernetes.io/projected/3823fc1b-a60f-4315-ae97-db1bc2e6974b-kube-api-access-b92c2\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:22.609862 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.609753    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-plugins-config-volume" pod "8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6" (UID: "8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:22.611668 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.611649    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-kube-api-access-f4rn5" pod "8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6" (UID: "8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6"). InnerVolumeSpecName "kube-api-access-f4rn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:22.711495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.711417    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4rn5\" (UniqueName: \"kubernetes.io/projected/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-kube-api-access-f4rn5\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:22.711495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.711444    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:22.993243 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.993155    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:22.993488 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.993175    4925 generic.go:356] "Generic (PLEG): container finished" podID="3823fc1b-a60f-4315-ae97-db1bc2e6974b" containerID="5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e" exitCode=0
Oct 30 19:01:22.993610 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.993200    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t" event={"ID":"3823fc1b-a60f-4315-ae97-db1bc2e6974b","Type":"ContainerDied","Data":"5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e"}
Oct 30 19:01:22.993690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.993645    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t" event={"ID":"3823fc1b-a60f-4315-ae97-db1bc2e6974b","Type":"ContainerDied","Data":"abbcd3c482b66b6e39ae89936d6a1c0b025a1d6eced1cf0a15867ed055927966"}
Oct 30 19:01:22.993690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.993680    4925 scope.go:122] "RemoveContainer" containerID="5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e"
Oct 30 19:01:22.993800 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:22.993267    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"
Oct 30 19:01:23.001411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.001381    4925 generic.go:356] "Generic (PLEG): container finished" podID="8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6" containerID="1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb" exitCode=0
Oct 30 19:01:23.002842 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.002224    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w" event={"ID":"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6","Type":"ContainerDied","Data":"1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb"}
Oct 30 19:01:23.002842 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.002276    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w" event={"ID":"8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6","Type":"ContainerDied","Data":"3c7422fefc531696c8f0a77d4735dd675646173f580af64971811fba435b51c2"}
Oct 30 19:01:23.003311 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.003174    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"
Oct 30 19:01:23.018601 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.018580    4925 scope.go:122] "RemoveContainer" containerID="5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e"
Oct 30 19:01:23.018963 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:01:23.018932    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e\": container with ID starting with 5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e not found: ID does not exist" containerID="5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e"
Oct 30 19:01:23.019045 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.018971    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e"} err="failed to get container status \"5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e\": rpc error: code = NotFound desc = could not find container \"5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e\": container with ID starting with 5ce29693b2a728c1638576906f7d085e77fe6a1a70597fe71caf9e44a521c68e not found: ID does not exist"
Oct 30 19:01:23.019045 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.018997    4925 scope.go:122] "RemoveContainer" containerID="1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb"
Oct 30 19:01:23.029432 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.029415    4925 scope.go:122] "RemoveContainer" containerID="1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb"
Oct 30 19:01:23.029712 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:01:23.029691    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb\": container with ID starting with 1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb not found: ID does not exist" containerID="1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb"
Oct 30 19:01:23.029806 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.029716    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb"} err="failed to get container status \"1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb\": rpc error: code = NotFound desc = could not find container \"1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb\": container with ID starting with 1c775fd8a2054799c07534ca75869e2edd1da4f1f3ffb2aa8ee68ffd152a8afb not found: ID does not exist"
Oct 30 19:01:23.041808 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.041786    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"]
Oct 30 19:01:23.044986 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.044963    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-6a327a-679688748c-6nt2t"]
Oct 30 19:01:23.060856 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.060832    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"]
Oct 30 19:01:23.066075 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:23.066050    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-6a327a-695c9989b8-9h65w"]
Oct 30 19:01:24.423050 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:24.423019    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3823fc1b-a60f-4315-ae97-db1bc2e6974b" path="/var/lib/kubelet/pods/3823fc1b-a60f-4315-ae97-db1bc2e6974b/volumes"
Oct 30 19:01:24.423639 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:24.423621    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6" path="/var/lib/kubelet/pods/8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6/volumes"
Oct 30 19:01:24.945386 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:24.945345    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:25.185834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:25.185797    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:25.630173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:25.626080    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:26.409127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:26.395942    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:27.166692 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:27.166657    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:27.742564 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:27.742518    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"]
Oct 30 19:01:27.862027 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:27.861993    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:28.080203 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:28.064284    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9" podUID="80192118-9d8a-4340-b466-e4f24a1bb08f" containerName="maas-api" containerID="cri-o://06d2f0ded9a3e58241a056c3d41a423628376e39f5d29303ca8accbc6d49df89" gracePeriod=30
Oct 30 19:01:28.081530 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:28.081497    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:28.278742 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:28.278549    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:28.757122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:28.755783    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:28.764583 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:28.764543    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9" podUID="80192118-9d8a-4340-b466-e4f24a1bb08f" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.134:8443/health\": dial tcp 10.132.0.134:8443: connect: connection refused"
Oct 30 19:01:28.942081 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:28.942045    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:29.474123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:29.451029    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:29.530069 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:29.530033    4925 generic.go:356] "Generic (PLEG): container finished" podID="80192118-9d8a-4340-b466-e4f24a1bb08f" containerID="06d2f0ded9a3e58241a056c3d41a423628376e39f5d29303ca8accbc6d49df89" exitCode=0
Oct 30 19:01:29.530593 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:29.530490    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9" event={"ID":"80192118-9d8a-4340-b466-e4f24a1bb08f","Type":"ContainerDied","Data":"06d2f0ded9a3e58241a056c3d41a423628376e39f5d29303ca8accbc6d49df89"}
Oct 30 19:01:29.798173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:29.784690    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 19:01:29.868167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:29.849673    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-metrics-certs\") pod \"80192118-9d8a-4340-b466-e4f24a1bb08f\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") "
Oct 30 19:01:29.876120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:29.872263    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80192118-9d8a-4340-b466-e4f24a1bb08f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80192118-9d8a-4340-b466-e4f24a1bb08f-openshift-service-ca\") pod \"80192118-9d8a-4340-b466-e4f24a1bb08f\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") "
Oct 30 19:01:29.876387 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:29.876363    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-maas-api-tls\") pod \"80192118-9d8a-4340-b466-e4f24a1bb08f\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") "
Oct 30 19:01:29.876526 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:29.876510    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80192118-9d8a-4340-b466-e4f24a1bb08f-kube-api-access-64k9d\" (UniqueName: \"kubernetes.io/projected/80192118-9d8a-4340-b466-e4f24a1bb08f-kube-api-access-64k9d\") pod \"80192118-9d8a-4340-b466-e4f24a1bb08f\" (UID: \"80192118-9d8a-4340-b466-e4f24a1bb08f\") "
Oct 30 19:01:29.898120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:29.884732    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80192118-9d8a-4340-b466-e4f24a1bb08f-openshift-service-ca" pod "80192118-9d8a-4340-b466-e4f24a1bb08f" (UID: "80192118-9d8a-4340-b466-e4f24a1bb08f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:29.925279 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:29.925230    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-maas-api-tls" pod "80192118-9d8a-4340-b466-e4f24a1bb08f" (UID: "80192118-9d8a-4340-b466-e4f24a1bb08f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:01:30.002800 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:29.981206    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80192118-9d8a-4340-b466-e4f24a1bb08f-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:30.002800 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:29.981241    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:30.094178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.070274    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-metrics-certs" pod "80192118-9d8a-4340-b466-e4f24a1bb08f" (UID: "80192118-9d8a-4340-b466-e4f24a1bb08f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:01:30.094178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.082624    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80192118-9d8a-4340-b466-e4f24a1bb08f-kube-api-access-64k9d" pod "80192118-9d8a-4340-b466-e4f24a1bb08f" (UID: "80192118-9d8a-4340-b466-e4f24a1bb08f"). InnerVolumeSpecName "kube-api-access-64k9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:30.094178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.083177    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:01:30.123135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.119292    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80192118-9d8a-4340-b466-e4f24a1bb08f-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:30.123135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.119324    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64k9d\" (UniqueName: \"kubernetes.io/projected/80192118-9d8a-4340-b466-e4f24a1bb08f-kube-api-access-64k9d\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:30.485711 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.485665    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 19:01:30.539126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.521274    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:01:30.702122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.698689    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:30.808594 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.808549    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9" event={"ID":"80192118-9d8a-4340-b466-e4f24a1bb08f","Type":"ContainerDied","Data":"971bf1e2182c4223a16f4a673432e7655f33d4d71e1f19e48416821de6163745"}
Oct 30 19:01:30.808952 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.808931    4925 scope.go:122] "RemoveContainer" containerID="06d2f0ded9a3e58241a056c3d41a423628376e39f5d29303ca8accbc6d49df89"
Oct 30 19:01:30.809316 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.809294    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"
Oct 30 19:01:30.869578 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.869543    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:01:30.962222 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.962160    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:01:30.977314 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:30.965313    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 19:01:31.208166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:31.201820    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:31.220701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:31.220656    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 19:01:31.598199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:31.588862    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:31.774123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:31.771365    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"]
Oct 30 19:01:32.049123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.032303    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:32.567123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.173892    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:32.594122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.195498    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:01:32.594328 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.214318    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:01:32.624634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.255008    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6a327a-7c6774784f-p7dp9"]
Oct 30 19:01:32.624634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.607321    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 19:01:32.624634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.317208    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:32.624634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.610673    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:32.624634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.518797    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:32.624634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.611063    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:32.873263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.873179    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80192118-9d8a-4340-b466-e4f24a1bb08f" path="/var/lib/kubelet/pods/80192118-9d8a-4340-b466-e4f24a1bb08f/volumes"
Oct 30 19:01:32.902122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.875935    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 19:01:32.904156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:32.904130    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 19:01:33.078125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:33.016378    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 19:01:33.385550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:33.331813    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 19:01:33.587249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:33.574242    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:01:33.588977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:33.588939    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:33.610700 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:33.609438    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:33.617454 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:33.612718    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:33.858511 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:33.858472    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:34.021468 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:34.019365    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 19:01:34.021468 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:34.020616    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 19:01:34.056153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:34.055671    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 19:01:34.061926 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:34.061850    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:01:34.790144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:34.782620    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:35.566870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:35.563672    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:36.478338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:36.478273    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:37.215319 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:37.215276    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:38.224493 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:38.224457    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:39.086914 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:39.086787    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:39.688150 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:39.688112    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:40.476630 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:40.476598    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:41.087049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:41.084387    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:41.774588 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:41.774554    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:42.513971 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:42.513934    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:43.157243 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:43.157205    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:43.381140 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:01:43.380314    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:01:43.917851 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:43.917811    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:45.468550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:45.468501    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:45.649407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:45.649363    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:46.594072 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:46.594029    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:46.606227 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:46.606182    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:46.607504 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:46.607475    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:46.620388 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:46.620356    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:46.652207 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:46.652169    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 30 19:01:46.653410 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:46.653385    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:01:47.292911 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.292861    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:47.293265 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.293233    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:47.610191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.585595    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.165:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:01:47.610191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.585725    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.165:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:47.674473 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.621233    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:47.674473 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.622492    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:47.674473 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.622672    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 19:01:47.674473 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.622757    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 19:01:47.674473 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.629493    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:47.674473 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.630158    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 19:01:47.743498 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.679595    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:47.743498 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.679952    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 19:01:47.743498 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:47.680575    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:01:48.221313 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:48.220123    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:01:48.238742 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:48.238254    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 19:01:48.337750 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:48.337715    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 19:01:48.416235 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:48.416204    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 19:01:48.455967 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:48.451409    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 19:01:48.797558 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:48.797522    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:49.586252 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:49.586211    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:50.289460 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.289063    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:50.688458 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.688421    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:50.874316 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.874276    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:50.995256 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.995162    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"]
Oct 30 19:01:50.996226 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.996199    4925 cpu_manager.go:414] "containerMap: removing container" podUID="8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6" containerName="payload-processing"
Oct 30 19:01:50.996226 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.996224    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6" containerName="payload-processing"
Oct 30 19:01:50.996447 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.996287    4925 cpu_manager.go:414] "containerMap: removing container" podUID="80192118-9d8a-4340-b466-e4f24a1bb08f" containerName="maas-api"
Oct 30 19:01:50.996447 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.996295    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80192118-9d8a-4340-b466-e4f24a1bb08f" containerName="maas-api"
Oct 30 19:01:50.996447 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.996310    4925 cpu_manager.go:414] "containerMap: removing container" podUID="3823fc1b-a60f-4315-ae97-db1bc2e6974b" containerName="payload-pre-processing"
Oct 30 19:01:50.996447 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.996319    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3823fc1b-a60f-4315-ae97-db1bc2e6974b" containerName="payload-pre-processing"
Oct 30 19:01:50.996665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.996467    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="80192118-9d8a-4340-b466-e4f24a1bb08f" containerName="maas-api"
Oct 30 19:01:50.996665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.996480    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="3823fc1b-a60f-4315-ae97-db1bc2e6974b" containerName="payload-pre-processing"
Oct 30 19:01:50.996665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:50.996500    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b86ce5f-c95f-4ad6-a785-71c8a5e6aaa6" containerName="payload-processing"
Oct 30 19:01:51.000453 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.000433    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.006804 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.006777    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-ef6b56-openshift-default-dockercfg-r79ls\""
Oct 30 19:01:51.009617 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.009295    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"]
Oct 30 19:01:51.056978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.056946    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-socket\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.057139 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.057015    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-488bh\" (UniqueName: \"kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-kube-api-access-488bh\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.057194 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.057134    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-credential-socket\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.057194 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.057166    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-podinfo\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.057268 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.057235    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-envoy\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.057306 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.057263    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-certs\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.057383 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.057359    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-token\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.057435 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.057420    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-data\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.057537 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.057521    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istiod-ca-cert\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.084822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.084785    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"]
Oct 30 19:01:51.090348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.090326    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.101630 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.101600    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:51.104249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.104226    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"]
Oct 30 19:01:51.106218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.106188    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-a1d55d-openshift-default-dockercfg-qstk8\""
Oct 30 19:01:51.159495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.159469    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-socket\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.159665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.159503    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-data\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.159665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.159623    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-certs\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.159783 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.159719    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-token\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.159783 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.159753    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istiod-ca-cert\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.159899 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.159793    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-podinfo\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.159899 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.159815    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb6qb\" (UniqueName: \"kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-kube-api-access-sb6qb\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.159899 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.159857    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-socket\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.160060 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.159925    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-data\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.160060 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.159976    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-credential-socket\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.160060 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.160005    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-envoy\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.160060 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.160053    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-488bh\" (UniqueName: \"kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-kube-api-access-488bh\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.160285 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.160231    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-socket\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.160285 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.160276    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-credential-socket\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.160392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.160295    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5760e51-b421-4e2b-8896-e61d63ef0e18-istiod-ca-cert\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.160392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.160314    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-podinfo\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.160499 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.160421    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-data\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.160499 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.160484    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-envoy\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.160608 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.160538    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-certs\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.160608 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.160564    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-credential-socket\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.160725 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.160627    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-token\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.160840 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.160818    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-certs\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.161049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.161031    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istiod-ca-cert\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.163685 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.163666    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-podinfo\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.163828 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.163812    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-envoy\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.168795 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.168733    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-token\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.169017 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.168993    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-488bh\" (UniqueName: \"kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-kube-api-access-488bh\") pod \"e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") " pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.262067    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-certs\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.262218    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-token\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.262266    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-podinfo\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.262295    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb6qb\" (UniqueName: \"kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-kube-api-access-sb6qb\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.262352    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-credential-socket\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.262377    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-envoy\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.262476    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5760e51-b421-4e2b-8896-e61d63ef0e18-istiod-ca-cert\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.262525    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-data\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.262641    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-socket\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.263115    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-socket\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.263385    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-certs\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.279529    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5760e51-b421-4e2b-8896-e61d63ef0e18-istiod-ca-cert\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.279778    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-data\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.280015    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-credential-socket\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.296178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.290403    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-envoy\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.297452 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.297292    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-podinfo\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.304906 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.304837    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb6qb\" (UniqueName: \"kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-kube-api-access-sb6qb\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.315695 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.315666    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-token\") pod \"e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") " pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.325717 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.325689    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:51.335695 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.335358    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:51.420748 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.420444    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:51.567375 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.567345    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:51.858519 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:01:51.858446    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5760e51_b421_4e2b_8896_e61d63ef0e18.slice/crio-52fbb4c86a958e969db5d5d6fc887858a2c93b3c20006e5e639a793d3cddb063 WatchSource:0}: Error finding container 52fbb4c86a958e969db5d5d6fc887858a2c93b3c20006e5e639a793d3cddb063: Status 404 returned error can't find the container with id 52fbb4c86a958e969db5d5d6fc887858a2c93b3c20006e5e639a793d3cddb063
Oct 30 19:01:51.858605 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.858536    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"]
Oct 30 19:01:51.861422 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:01:51.861396    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae0e1c00_9ab0_4e0b_911b_31e1843d9232.slice/crio-a6a982921fb216892614e4d3958e6c30183fddc3a986f594b82936010aee0ab5 WatchSource:0}: Error finding container a6a982921fb216892614e4d3958e6c30183fddc3a986f594b82936010aee0ab5: Status 404 returned error can't find the container with id a6a982921fb216892614e4d3958e6c30183fddc3a986f594b82936010aee0ab5
Oct 30 19:01:51.861534 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.861406    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:51.861534 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.861489    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:51.861616 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.861534    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:51.862440 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.862420    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"]
Oct 30 19:01:51.863502 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.863478    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:51.864016 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.863989    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:51.864116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.864049    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:51.864116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:51.864079    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:52.268598 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:52.268566    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:52.482202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:52.482068    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" event={"ID":"f5760e51-b421-4e2b-8896-e61d63ef0e18","Type":"ContainerStarted","Data":"5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8"}
Oct 30 19:01:52.482202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:52.482150    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" event={"ID":"f5760e51-b421-4e2b-8896-e61d63ef0e18","Type":"ContainerStarted","Data":"52fbb4c86a958e969db5d5d6fc887858a2c93b3c20006e5e639a793d3cddb063"}
Oct 30 19:01:52.484332 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:52.484300    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" event={"ID":"ae0e1c00-9ab0-4e0b-911b-31e1843d9232","Type":"ContainerStarted","Data":"45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0"}
Oct 30 19:01:52.484467 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:52.484353    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" event={"ID":"ae0e1c00-9ab0-4e0b-911b-31e1843d9232","Type":"ContainerStarted","Data":"a6a982921fb216892614e4d3958e6c30183fddc3a986f594b82936010aee0ab5"}
Oct 30 19:01:52.505041 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:52.504998    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" podStartSLOduration=1.504985118 podStartE2EDuration="1.504985118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:52.50243959 +0000 UTC m=+1534.218784666" watchObservedRunningTime="2026-10-30 19:01:52.504985118 +0000 UTC m=+1534.221330194"
Oct 30 19:01:52.527538 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:52.527453    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" podStartSLOduration=2.527441561 podStartE2EDuration="2.527441561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:52.52428837 +0000 UTC m=+1534.240633444" watchObservedRunningTime="2026-10-30 19:01:52.527441561 +0000 UTC m=+1534.243786634"
Oct 30 19:01:53.077123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:53.072363    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:53.336342 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:53.336269    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:53.343298 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:53.343273    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:53.436278 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:53.436243    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:53.442901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:53.442872    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:53.493407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:53.493380    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:53.493882 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:53.493430    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:53.494521 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:53.494498    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:01:53.495572 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:53.495545    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:01:53.687138 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:53.678784    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:54.520995 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:54.520920    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:54.918563 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:54.918532    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:55.121884 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:55.121848    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:55.308692 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:55.308597    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:56.096665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.096627    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"]
Oct 30 19:01:56.102340 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.102315    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"
Oct 30 19:01:56.116764 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.116736    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w4-ef6b56\""
Oct 30 19:01:56.130330 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.130302    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"]
Oct 30 19:01:56.171161 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.171121    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpznf\" (UniqueName: \"kubernetes.io/projected/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-kube-api-access-qpznf\") pod \"payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd\" (UID: \"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"
Oct 30 19:01:56.171363 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.171198    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd\" (UID: \"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"
Oct 30 19:01:56.186196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.186040    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"]
Oct 30 19:01:56.191673 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.191642    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:56.196014 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.195983    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"
Oct 30 19:01:56.200874 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.199260    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"]
Oct 30 19:01:56.209834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.209805    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w4-ef6b56-dockercfg-plnmh\""
Oct 30 19:01:56.328123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.313710    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx\" (UID: \"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"
Oct 30 19:01:56.328123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.313763    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8vnd\" (UniqueName: \"kubernetes.io/projected/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-kube-api-access-l8vnd\") pod \"payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx\" (UID: \"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"
Oct 30 19:01:56.328123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.314027    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpznf\" (UniqueName: \"kubernetes.io/projected/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-kube-api-access-qpznf\") pod \"payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd\" (UID: \"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"
Oct 30 19:01:56.328123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.314078    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd\" (UID: \"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"
Oct 30 19:01:56.464218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.394029    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd\" (UID: \"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"
Oct 30 19:01:56.470994 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.465328    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx\" (UID: \"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"
Oct 30 19:01:56.483680 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.474179    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8vnd\" (UniqueName: \"kubernetes.io/projected/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-kube-api-access-l8vnd\") pod \"payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx\" (UID: \"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"
Oct 30 19:01:56.486971 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.483982    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx\" (UID: \"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"
Oct 30 19:01:56.540151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.539789    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:56.564125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.546142    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8vnd\" (UniqueName: \"kubernetes.io/projected/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-kube-api-access-l8vnd\") pod \"payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx\" (UID: \"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"
Oct 30 19:01:56.589340 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.589306    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpznf\" (UniqueName: \"kubernetes.io/projected/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-kube-api-access-qpznf\") pod \"payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd\" (UID: \"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"
Oct 30 19:01:56.718953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.718866    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"
Oct 30 19:01:56.719173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.718963    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:56.824887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:56.824533    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"
Oct 30 19:01:57.020812 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:01:57.019977    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c157a5a_f699_4ea7_a5d7_7b4bb851ef88.slice/crio-af1c1d41000e5620d69e5ea1e8b7d76f2a362a10fe13bf98dff98537556c2ae2 WatchSource:0}: Error finding container af1c1d41000e5620d69e5ea1e8b7d76f2a362a10fe13bf98dff98537556c2ae2: Status 404 returned error can't find the container with id af1c1d41000e5620d69e5ea1e8b7d76f2a362a10fe13bf98dff98537556c2ae2
Oct 30 19:01:57.021007 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.020919    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"]
Oct 30 19:01:57.097457 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.097430    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"]
Oct 30 19:01:57.102770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.102747    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"
Oct 30 19:01:57.108191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.108150    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-a1d55d\""
Oct 30 19:01:57.112406 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.112384    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"]
Oct 30 19:01:57.119546 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.119224    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:57.126715 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.124003    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-a1d55d-dockercfg-bx7fk\""
Oct 30 19:01:57.133953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.133927    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"]
Oct 30 19:01:57.143854 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:01:57.143819    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86f2d7f0_ab22_4a23_81c7_63536b3fb2ad.slice/crio-d8ba748142a0927ab5003f325bd23c5223f77d64aa55782fca73b705cafff34d WatchSource:0}: Error finding container d8ba748142a0927ab5003f325bd23c5223f77d64aa55782fca73b705cafff34d: Status 404 returned error can't find the container with id d8ba748142a0927ab5003f325bd23c5223f77d64aa55782fca73b705cafff34d
Oct 30 19:01:57.149643 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.149623    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"]
Oct 30 19:01:57.149864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.149850    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"
Oct 30 19:01:57.167475 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.167456    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"]
Oct 30 19:01:57.258551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.258506    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cded1de5-2ce0-454c-8426-22558fd26da6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76\" (UID: \"cded1de5-2ce0-454c-8426-22558fd26da6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"
Oct 30 19:01:57.258900 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.258799    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b353a66-fb6e-418d-9538-bf279927ebfc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9\" (UID: \"0b353a66-fb6e-418d-9538-bf279927ebfc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"
Oct 30 19:01:57.259064 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.258927    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvvcj\" (UniqueName: \"kubernetes.io/projected/0b353a66-fb6e-418d-9538-bf279927ebfc-kube-api-access-cvvcj\") pod \"payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9\" (UID: \"0b353a66-fb6e-418d-9538-bf279927ebfc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"
Oct 30 19:01:57.259064 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.258992    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h59dh\" (UniqueName: \"kubernetes.io/projected/cded1de5-2ce0-454c-8426-22558fd26da6-kube-api-access-h59dh\") pod \"payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76\" (UID: \"cded1de5-2ce0-454c-8426-22558fd26da6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"
Oct 30 19:01:57.310231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.306475    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:57.374503 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.374377    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b353a66-fb6e-418d-9538-bf279927ebfc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9\" (UID: \"0b353a66-fb6e-418d-9538-bf279927ebfc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"
Oct 30 19:01:57.374884 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.374862    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvvcj\" (UniqueName: \"kubernetes.io/projected/0b353a66-fb6e-418d-9538-bf279927ebfc-kube-api-access-cvvcj\") pod \"payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9\" (UID: \"0b353a66-fb6e-418d-9538-bf279927ebfc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"
Oct 30 19:01:57.375044 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.375027    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h59dh\" (UniqueName: \"kubernetes.io/projected/cded1de5-2ce0-454c-8426-22558fd26da6-kube-api-access-h59dh\") pod \"payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76\" (UID: \"cded1de5-2ce0-454c-8426-22558fd26da6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"
Oct 30 19:01:57.375217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.375199    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cded1de5-2ce0-454c-8426-22558fd26da6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76\" (UID: \"cded1de5-2ce0-454c-8426-22558fd26da6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"
Oct 30 19:01:57.376039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.375438    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b353a66-fb6e-418d-9538-bf279927ebfc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9\" (UID: \"0b353a66-fb6e-418d-9538-bf279927ebfc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"
Oct 30 19:01:57.376508 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.376391    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cded1de5-2ce0-454c-8426-22558fd26da6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76\" (UID: \"cded1de5-2ce0-454c-8426-22558fd26da6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"
Oct 30 19:01:57.442439 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.439263    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvvcj\" (UniqueName: \"kubernetes.io/projected/0b353a66-fb6e-418d-9538-bf279927ebfc-kube-api-access-cvvcj\") pod \"payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9\" (UID: \"0b353a66-fb6e-418d-9538-bf279927ebfc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"
Oct 30 19:01:57.452004 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.451965    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h59dh\" (UniqueName: \"kubernetes.io/projected/cded1de5-2ce0-454c-8426-22558fd26da6-kube-api-access-h59dh\") pod \"payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76\" (UID: \"cded1de5-2ce0-454c-8426-22558fd26da6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"
Oct 30 19:01:57.491803 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.491345    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"
Oct 30 19:01:57.606030 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.602968    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx" event={"ID":"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad","Type":"ContainerStarted","Data":"c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892"}
Oct 30 19:01:57.606030 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.603069    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx" event={"ID":"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad","Type":"ContainerStarted","Data":"d8ba748142a0927ab5003f325bd23c5223f77d64aa55782fca73b705cafff34d"}
Oct 30 19:01:57.606030 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.604027    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"
Oct 30 19:01:57.606570 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.606543    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd" event={"ID":"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88","Type":"ContainerStarted","Data":"18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4"}
Oct 30 19:01:57.606616 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.606596    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd" event={"ID":"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88","Type":"ContainerStarted","Data":"af1c1d41000e5620d69e5ea1e8b7d76f2a362a10fe13bf98dff98537556c2ae2"}
Oct 30 19:01:57.606773 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.606757    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"
Oct 30 19:01:57.633172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.628215    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx" podStartSLOduration=1.628199489 podStartE2EDuration="1.628199489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:57.620140194 +0000 UTC m=+1539.336485271" watchObservedRunningTime="2026-10-30 19:01:57.628199489 +0000 UTC m=+1539.344544566"
Oct 30 19:01:57.641557 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.641516    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd" podStartSLOduration=1.641505804 podStartE2EDuration="1.641505804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:57.638131599 +0000 UTC m=+1539.354476685" watchObservedRunningTime="2026-10-30 19:01:57.641505804 +0000 UTC m=+1539.357850879"
Oct 30 19:01:57.730994 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.730969    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"
Oct 30 19:01:57.799531 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:01:57.799495    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcded1de5_2ce0_454c_8426_22558fd26da6.slice/crio-efeed0c61449eee382c5eeea5b544bcdc6a64f7bacf3b0b9cac422cf46de08da WatchSource:0}: Error finding container efeed0c61449eee382c5eeea5b544bcdc6a64f7bacf3b0b9cac422cf46de08da: Status 404 returned error can't find the container with id efeed0c61449eee382c5eeea5b544bcdc6a64f7bacf3b0b9cac422cf46de08da
Oct 30 19:01:57.803541 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.803487    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"]
Oct 30 19:01:57.893456 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:57.893422    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:58.029134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:58.029106    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"]
Oct 30 19:01:58.030401 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:01:58.030369    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b353a66_fb6e_418d_9538_bf279927ebfc.slice/crio-b5d84391dbaa25e98dd334894cc331c618907f5c9424202bfb761a54b575e8fb WatchSource:0}: Error finding container b5d84391dbaa25e98dd334894cc331c618907f5c9424202bfb761a54b575e8fb: Status 404 returned error can't find the container with id b5d84391dbaa25e98dd334894cc331c618907f5c9424202bfb761a54b575e8fb
Oct 30 19:01:58.602113 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:58.602061    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:58.616010 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:58.615971    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76" event={"ID":"cded1de5-2ce0-454c-8426-22558fd26da6","Type":"ContainerStarted","Data":"9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3"}
Oct 30 19:01:58.616195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:58.616042    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76" event={"ID":"cded1de5-2ce0-454c-8426-22558fd26da6","Type":"ContainerStarted","Data":"efeed0c61449eee382c5eeea5b544bcdc6a64f7bacf3b0b9cac422cf46de08da"}
Oct 30 19:01:58.616195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:58.616074    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"
Oct 30 19:01:58.618462 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:58.618429    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9" event={"ID":"0b353a66-fb6e-418d-9538-bf279927ebfc","Type":"ContainerStarted","Data":"3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706"}
Oct 30 19:01:58.618728 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:58.618713    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"
Oct 30 19:01:58.618786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:58.618747    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9" event={"ID":"0b353a66-fb6e-418d-9538-bf279927ebfc","Type":"ContainerStarted","Data":"b5d84391dbaa25e98dd334894cc331c618907f5c9424202bfb761a54b575e8fb"}
Oct 30 19:01:58.636840 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:58.636798    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76" podStartSLOduration=1.6367859729999998 podStartE2EDuration="1.636785973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:58.632952926 +0000 UTC m=+1540.349298003" watchObservedRunningTime="2026-10-30 19:01:58.636785973 +0000 UTC m=+1540.353131048"
Oct 30 19:01:58.653916 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:58.653876    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9" podStartSLOduration=1.653863495 podStartE2EDuration="1.653863495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:58.651081001 +0000 UTC m=+1540.367426078" watchObservedRunningTime="2026-10-30 19:01:58.653863495 +0000 UTC m=+1540.370208570"
Oct 30 19:01:59.303516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:59.303486    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:59.989181 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:01:59.987846    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:00.631758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.629740    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"]
Oct 30 19:02:00.645898 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.639761    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.649126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.646514    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w4-ef6b56-serving-cert\""
Oct 30 19:02:00.655680 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.655298    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"]
Oct 30 19:02:00.761266 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.761234    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:00.780492 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.780468    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.780603 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.780499    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.780657 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.780628    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.780778 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.780764    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2bgs\" (UniqueName: \"kubernetes.io/projected/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-kube-api-access-x2bgs\") pod \"maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.882424 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.882193    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2bgs\" (UniqueName: \"kubernetes.io/projected/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-kube-api-access-x2bgs\") pod \"maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.886777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.886744    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.887946 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.887920    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.889959 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.889932    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.899355 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.899257    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.903941 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.903868    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.905125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.904596    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2bgs\" (UniqueName: \"kubernetes.io/projected/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-kube-api-access-x2bgs\") pod \"maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.906892 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.906870    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:00.952305 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.952276    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:00.995794 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:00.995767    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:01.348650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.348598    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"]
Oct 30 19:02:01.371058 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:02:01.367049    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e6a4811_2f5c_4fc8_9d66_e85a44e7dc76.slice/crio-8bca601faa9c2e855655a7f72c6feddf91dc32af697ca6a66db910fb50217aa3 WatchSource:0}: Error finding container 8bca601faa9c2e855655a7f72c6feddf91dc32af697ca6a66db910fb50217aa3: Status 404 returned error can't find the container with id 8bca601faa9c2e855655a7f72c6feddf91dc32af697ca6a66db910fb50217aa3
Oct 30 19:02:01.431454 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.431422    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:01.461561 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.461529    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"]
Oct 30 19:02:01.469201 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.469180    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.475539 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.475515    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-a1d55d-serving-cert\""
Oct 30 19:02:01.480726 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.480547    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"]
Oct 30 19:02:01.633728 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.633641    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:01.676039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.676003    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.677954 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.677925    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93cec944-2c6d-4e13-8fad-c663bd7934ae-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.680499 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.680477    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.681365 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.681343    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2tv5\" (UniqueName: \"kubernetes.io/projected/93cec944-2c6d-4e13-8fad-c663bd7934ae-kube-api-access-q2tv5\") pod \"maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.715022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.714668    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m" event={"ID":"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76","Type":"ContainerStarted","Data":"8bca601faa9c2e855655a7f72c6feddf91dc32af697ca6a66db910fb50217aa3"}
Oct 30 19:02:01.783672 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.783620    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93cec944-2c6d-4e13-8fad-c663bd7934ae-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.783873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.783690    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.783873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.783848    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2tv5\" (UniqueName: \"kubernetes.io/projected/93cec944-2c6d-4e13-8fad-c663bd7934ae-kube-api-access-q2tv5\") pod \"maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.784055 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.784034    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.787207 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.787175    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93cec944-2c6d-4e13-8fad-c663bd7934ae-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.789680 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.789652    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.793221 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.793188    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.802918 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.802818    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2tv5\" (UniqueName: \"kubernetes.io/projected/93cec944-2c6d-4e13-8fad-c663bd7934ae-kube-api-access-q2tv5\") pod \"maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:01.839857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:01.839814    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:02.033218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:02.033182    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:02.108120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:02.107498    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:02.747082 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:02.746999    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:02.752885 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:02.752855    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m" event={"ID":"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76","Type":"ContainerStarted","Data":"e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5"}
Oct 30 19:02:02.753068 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:02.752982    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:02.765498 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:02.765470    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"]
Oct 30 19:02:02.774150 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:02:02.774118    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93cec944_2c6d_4e13_8fad_c663bd7934ae.slice/crio-cc684cff7eb84fa05c55c9d9e263517dc5ef052d3bf2ff12c3f22ab127867acc WatchSource:0}: Error finding container cc684cff7eb84fa05c55c9d9e263517dc5ef052d3bf2ff12c3f22ab127867acc: Status 404 returned error can't find the container with id cc684cff7eb84fa05c55c9d9e263517dc5ef052d3bf2ff12c3f22ab127867acc
Oct 30 19:02:02.777174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:02.777123    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m" podStartSLOduration=2.222554226 podStartE2EDuration="2.777088647s" totalImagesPullingTime="554.534421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:02.772364871 +0000 UTC m=+1544.488709947" watchObservedRunningTime="2026-10-30 19:02:02.777088647 +0000 UTC m=+1544.493433723"
Oct 30 19:02:03.678052 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:03.678019    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:03.762567 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:03.762533    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8" event={"ID":"93cec944-2c6d-4e13-8fad-c663bd7934ae","Type":"ContainerStarted","Data":"d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711"}
Oct 30 19:02:03.762765 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:03.762592    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8" event={"ID":"93cec944-2c6d-4e13-8fad-c663bd7934ae","Type":"ContainerStarted","Data":"cc684cff7eb84fa05c55c9d9e263517dc5ef052d3bf2ff12c3f22ab127867acc"}
Oct 30 19:02:03.762892 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:03.762876    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:03.781566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:03.781514    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8" podStartSLOduration=2.289855783 podStartE2EDuration="2.781497529s" totalImagesPullingTime="491.641746ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:03.781201828 +0000 UTC m=+1545.497546905" watchObservedRunningTime="2026-10-30 19:02:03.781497529 +0000 UTC m=+1545.497842607"
Oct 30 19:02:05.435141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:05.434844    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:05.898807 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:05.898719    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:06.245225 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:06.245191    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:06.344121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:06.343339    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:06.467123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:06.451373    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:06.467123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:06.455251    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:06.689122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:06.688256    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:06.811122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:06.810324    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:07.316147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:07.275255    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:07.316147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:07.292056    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 19:02:07.367131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:07.366507    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:02:07.367337 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:07.367146    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 19:02:07.427136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:07.425016    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded"
Oct 30 19:02:07.485129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:07.458171    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:07.485129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:07.458884    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 19:02:07.545764 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:07.539067    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:07.700147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:07.689612    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:07.733414 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:07.733384    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:02:08.068008 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.067906    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:02:08.068698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.068663    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:08.301232 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.301185    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": context deadline exceeded"
Oct 30 19:02:08.302784 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.302760    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 19:02:08.368314 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.368212    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:02:08.410121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.397382    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 19:02:08.500121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.476248    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:02:08.537167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.501313    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:02:08.537167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.501393    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:08.656617 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.656516    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:08.688313 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.688285    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 19:02:08.688583 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.688567    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:02:08.688754 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.688739    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"
Oct 30 19:02:08.688941 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.688908    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 19:02:08.743742 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.743685    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 19:02:08.786057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.764746    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 19:02:08.790576 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.790545    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:08.806410 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.806379    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:02:08.817299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.817274    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"]
Oct 30 19:02:08.875986 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.875960    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"
Oct 30 19:02:08.885868 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.885820    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-c2e3f9-dockercfg-hhct7\""
Oct 30 19:02:08.895459 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.886675    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-c2e3f9\""
Oct 30 19:02:08.895678 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.895054    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"]
Oct 30 19:02:08.913190 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.911871    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"]
Oct 30 19:02:08.918609 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.918573    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"]
Oct 30 19:02:08.918769 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.918685    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"
Oct 30 19:02:08.969602 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.969576    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:08.985055 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.985028    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjpzr\" (UniqueName: \"kubernetes.io/projected/3107703b-0cfb-4320-84c9-d023eeea6c36-kube-api-access-rjpzr\") pod \"payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh\" (UID: \"3107703b-0cfb-4320-84c9-d023eeea6c36\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"
Oct 30 19:02:08.985473 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:08.985447    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3107703b-0cfb-4320-84c9-d023eeea6c36-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh\" (UID: \"3107703b-0cfb-4320-84c9-d023eeea6c36\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"
Oct 30 19:02:09.095850 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.093215    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xphpz\" (UniqueName: \"kubernetes.io/projected/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-kube-api-access-xphpz\") pod \"payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg\" (UID: \"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"
Oct 30 19:02:09.095850 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.093316    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3107703b-0cfb-4320-84c9-d023eeea6c36-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh\" (UID: \"3107703b-0cfb-4320-84c9-d023eeea6c36\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"
Oct 30 19:02:09.095850 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.093377    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjpzr\" (UniqueName: \"kubernetes.io/projected/3107703b-0cfb-4320-84c9-d023eeea6c36-kube-api-access-rjpzr\") pod \"payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh\" (UID: \"3107703b-0cfb-4320-84c9-d023eeea6c36\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"
Oct 30 19:02:09.095850 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.093568    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg\" (UID: \"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"
Oct 30 19:02:09.095850 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.094732    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3107703b-0cfb-4320-84c9-d023eeea6c36-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh\" (UID: \"3107703b-0cfb-4320-84c9-d023eeea6c36\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"
Oct 30 19:02:09.165186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.161704    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjpzr\" (UniqueName: \"kubernetes.io/projected/3107703b-0cfb-4320-84c9-d023eeea6c36-kube-api-access-rjpzr\") pod \"payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh\" (UID: \"3107703b-0cfb-4320-84c9-d023eeea6c36\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"
Oct 30 19:02:09.196782 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.195441    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg\" (UID: \"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"
Oct 30 19:02:09.196782 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.195667    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xphpz\" (UniqueName: \"kubernetes.io/projected/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-kube-api-access-xphpz\") pod \"payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg\" (UID: \"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"
Oct 30 19:02:09.197444 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.197367    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg\" (UID: \"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"
Oct 30 19:02:09.214575 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.214398    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xphpz\" (UniqueName: \"kubernetes.io/projected/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-kube-api-access-xphpz\") pod \"payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg\" (UID: \"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"
Oct 30 19:02:09.249662 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.249623    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"
Oct 30 19:02:09.250269 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.250126    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"
Oct 30 19:02:09.387131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.382691    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:09.627632 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.627554    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"
Oct 30 19:02:09.627632 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.627624    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"
Oct 30 19:02:09.628249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.627663    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"
Oct 30 19:02:09.721519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.721490    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"]
Oct 30 19:02:09.723562 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:02:09.723533    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3107703b_0cfb_4320_84c9_d023eeea6c36.slice/crio-cad87d1fb29418700054e3ca730bcac7e2a05d16d9a7f12cb73b7c46c51ed6bc WatchSource:0}: Error finding container cad87d1fb29418700054e3ca730bcac7e2a05d16d9a7f12cb73b7c46c51ed6bc: Status 404 returned error can't find the container with id cad87d1fb29418700054e3ca730bcac7e2a05d16d9a7f12cb73b7c46c51ed6bc
Oct 30 19:02:09.742191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.742137    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"]
Oct 30 19:02:09.743352 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:02:09.743328    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4ba1e6_4f4c_42c6_bc34_4eafcc18e428.slice/crio-234052df403dde5dc5b4a64085279f9d434424b4f1d3b4dc39b73f5cca25e678 WatchSource:0}: Error finding container 234052df403dde5dc5b4a64085279f9d434424b4f1d3b4dc39b73f5cca25e678: Status 404 returned error can't find the container with id 234052df403dde5dc5b4a64085279f9d434424b4f1d3b4dc39b73f5cca25e678
Oct 30 19:02:09.812560 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:09.812535    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:02:10.026416 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:10.026328    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:10.229080 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:10.229044    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh" event={"ID":"3107703b-0cfb-4320-84c9-d023eeea6c36","Type":"ContainerStarted","Data":"ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3"}
Oct 30 19:02:10.229298 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:10.229123    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh" event={"ID":"3107703b-0cfb-4320-84c9-d023eeea6c36","Type":"ContainerStarted","Data":"cad87d1fb29418700054e3ca730bcac7e2a05d16d9a7f12cb73b7c46c51ed6bc"}
Oct 30 19:02:10.229298 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:10.229162    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"
Oct 30 19:02:10.230891 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:10.230864    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg" event={"ID":"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428","Type":"ContainerStarted","Data":"b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d"}
Oct 30 19:02:10.231049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:10.230906    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg" event={"ID":"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428","Type":"ContainerStarted","Data":"234052df403dde5dc5b4a64085279f9d434424b4f1d3b4dc39b73f5cca25e678"}
Oct 30 19:02:10.231049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:10.230989    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"
Oct 30 19:02:10.244982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:10.244814    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh" podStartSLOduration=2.244798644 podStartE2EDuration="2.244798644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:10.244084127 +0000 UTC m=+1551.960429203" watchObservedRunningTime="2026-10-30 19:02:10.244798644 +0000 UTC m=+1551.961143721"
Oct 30 19:02:10.262547 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:10.262503    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg" podStartSLOduration=2.2624921159999998 podStartE2EDuration="2.262492116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:10.259639329 +0000 UTC m=+1551.975984417" watchObservedRunningTime="2026-10-30 19:02:10.262492116 +0000 UTC m=+1551.978837192"
Oct 30 19:02:10.278801 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:10.278729    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:10.485257 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:10.483289    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:11.059119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:11.058508    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:11.641831 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:11.641793    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:11.833310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:11.833264    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:12.468132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:12.465571    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:13.309219 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:02:13.309176    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:02:13.499122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:13.497749    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:14.692310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:14.692278    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:14.902122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:14.890712    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:15.092537 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.092436    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"]
Oct 30 19:02:15.097466 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.097376    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:15.100464 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.100437    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"]
Oct 30 19:02:15.100635 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.100586    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.106372 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.106202    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"]
Oct 30 19:02:15.106372 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.106253    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"]
Oct 30 19:02:15.106372 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.106334    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.174030 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.173994    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.174229 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174090    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.174300 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174226    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.174300 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174250    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040bb776-e34b-4301-8bd0-634c316c46c5-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.174300 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174287    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.174445 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174407    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.174445 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174429    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.174549 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174466    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.174609 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174586    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn5mj\" (UniqueName: \"kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-kube-api-access-hn5mj\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.174667 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174619    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dq5v\" (UniqueName: \"kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-kube-api-access-9dq5v\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.174720 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174669    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.174794 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174777    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.174859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174831    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040bb776-e34b-4301-8bd0-634c316c46c5-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.174925 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174881    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.174985 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.174925    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.175079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.175057    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.175151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.175115    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.175201 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.175169    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.276327 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276286    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.276516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276334    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.276516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276361    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.276516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276409    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.276516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276446    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.276790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276523    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.276790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276554    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040bb776-e34b-4301-8bd0-634c316c46c5-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.276790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276602    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.276790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276658    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.276790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276685    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.276790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276715    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.277171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276798    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn5mj\" (UniqueName: \"kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-kube-api-access-hn5mj\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.277171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276822    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dq5v\" (UniqueName: \"kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-kube-api-access-9dq5v\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.277171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276873    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.277171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276909    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.277171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.276985    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.277171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.277005    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-credential-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.277171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.277010    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.277171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.277081    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040bb776-e34b-4301-8bd0-634c316c46c5-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.277171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.277127    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.277817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.277181    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.277817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.277215    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.277817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.277256    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.277817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.277640    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-socket\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.277817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.277662    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.277817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.277745    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-data\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.278306 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.277982    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040bb776-e34b-4301-8bd0-634c316c46c5-istiod-ca-cert\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.278306 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.278136    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-certs\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.279905 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.279734    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.280021 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.279911    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040bb776-e34b-4301-8bd0-634c316c46c5-istio-podinfo\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.280173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.280150    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.280380 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.280352    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-envoy\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.284679 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.284653    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.286314 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.286292    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dq5v\" (UniqueName: \"kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-kube-api-access-9dq5v\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.286456 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.286416    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-istio-token\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.287247 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.287228    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn5mj\" (UniqueName: \"kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-kube-api-access-hn5mj\") pod \"e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") " pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.405945 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.405848    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:15.435567 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.435536    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:15.441061 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.441037    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:15.594299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.594084    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:15.767847 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:02:15.767807    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33bb4a2a_8455_46ae_b048_29f7ba08eef7.slice/crio-a751bb2c33a6330b7a4d015d60cea96374cfadf78604561d37a2e284d4235b0a WatchSource:0}: Error finding container a751bb2c33a6330b7a4d015d60cea96374cfadf78604561d37a2e284d4235b0a: Status 404 returned error can't find the container with id a751bb2c33a6330b7a4d015d60cea96374cfadf78604561d37a2e284d4235b0a
Oct 30 19:02:15.768907 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.768871    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"]
Oct 30 19:02:15.769440 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:02:15.769406    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod040bb776_e34b_4301_8bd0_634c316c46c5.slice/crio-c1e51db9c14663ddc6404862190622521c45ec4cf789f0d9cdf504c1657c231a WatchSource:0}: Error finding container c1e51db9c14663ddc6404862190622521c45ec4cf789f0d9cdf504c1657c231a: Status 404 returned error can't find the container with id c1e51db9c14663ddc6404862190622521c45ec4cf789f0d9cdf504c1657c231a
Oct 30 19:02:15.776115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.771871    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:15.776115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.771955    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:15.776115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.772000    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:15.776115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.774230    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"]
Oct 30 19:02:15.776115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.774554    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:15.776115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.774676    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:15.776115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.774723    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:15.855936 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:15.855625    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:16.045785 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.045706    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"]
Oct 30 19:02:16.051108 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.051072    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"
Oct 30 19:02:16.054151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.054122    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:16.054934 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.054912    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-baa504\""
Oct 30 19:02:16.055219 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.055199    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-baa504-dockercfg-pcngq\""
Oct 30 19:02:16.063702 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.063679    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"]
Oct 30 19:02:16.119813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.119774    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8afa108-8874-4821-9a23-784ff39e42ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq\" (UID: \"d8afa108-8874-4821-9a23-784ff39e42ce\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"
Oct 30 19:02:16.120006 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.119958    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmzd7\" (UniqueName: \"kubernetes.io/projected/d8afa108-8874-4821-9a23-784ff39e42ce-kube-api-access-pmzd7\") pod \"payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq\" (UID: \"d8afa108-8874-4821-9a23-784ff39e42ce\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"
Oct 30 19:02:16.242120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.225181    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmzd7\" (UniqueName: \"kubernetes.io/projected/d8afa108-8874-4821-9a23-784ff39e42ce-kube-api-access-pmzd7\") pod \"payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq\" (UID: \"d8afa108-8874-4821-9a23-784ff39e42ce\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"
Oct 30 19:02:16.242120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.225482    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8afa108-8874-4821-9a23-784ff39e42ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq\" (UID: \"d8afa108-8874-4821-9a23-784ff39e42ce\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"
Oct 30 19:02:16.242120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.226537    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8afa108-8874-4821-9a23-784ff39e42ce-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq\" (UID: \"d8afa108-8874-4821-9a23-784ff39e42ce\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"
Oct 30 19:02:16.266119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.250359    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:16.266119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.253788    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmzd7\" (UniqueName: \"kubernetes.io/projected/d8afa108-8874-4821-9a23-784ff39e42ce-kube-api-access-pmzd7\") pod \"payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq\" (UID: \"d8afa108-8874-4821-9a23-784ff39e42ce\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"
Oct 30 19:02:16.402360 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.402272    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"
Oct 30 19:02:16.465133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.453886    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused" start-of-body=
Oct 30 19:02:16.465133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.453942    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused"
Oct 30 19:02:16.506947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.506258    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:16.554085 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.516228    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" podStartSLOduration=1.516209576 podStartE2EDuration="1.516209576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:16.482256839 +0000 UTC m=+1558.198601916" watchObservedRunningTime="2026-10-30 19:02:16.516209576 +0000 UTC m=+1558.232554656"
Oct 30 19:02:16.578166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.574568    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" event={"ID":"33bb4a2a-8455-46ae-b048-29f7ba08eef7","Type":"ContainerStarted","Data":"32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e"}
Oct 30 19:02:16.578166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.574637    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" event={"ID":"33bb4a2a-8455-46ae-b048-29f7ba08eef7","Type":"ContainerStarted","Data":"a751bb2c33a6330b7a4d015d60cea96374cfadf78604561d37a2e284d4235b0a"}
Oct 30 19:02:16.578166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.574663    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" event={"ID":"040bb776-e34b-4301-8bd0-634c316c46c5","Type":"ContainerStarted","Data":"de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14"}
Oct 30 19:02:16.578166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.574703    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:16.578166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.574726    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" event={"ID":"040bb776-e34b-4301-8bd0-634c316c46c5","Type":"ContainerStarted","Data":"c1e51db9c14663ddc6404862190622521c45ec4cf789f0d9cdf504c1657c231a"}
Oct 30 19:02:16.602678 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.602604    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" podStartSLOduration=1.6025831099999999 podStartE2EDuration="1.60258311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:16.569155025 +0000 UTC m=+1558.285500104" watchObservedRunningTime="2026-10-30 19:02:16.60258311 +0000 UTC m=+1558.318928188"
Oct 30 19:02:16.920665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:16.920436    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"]
Oct 30 19:02:16.923156 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:02:16.923082    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8afa108_8874_4821_9a23_784ff39e42ce.slice/crio-15f2689fe8c3da1e5bcde2e9813107101f6c4bc45634b2d7f035969dba4c74ac WatchSource:0}: Error finding container 15f2689fe8c3da1e5bcde2e9813107101f6c4bc45634b2d7f035969dba4c74ac: Status 404 returned error can't find the container with id 15f2689fe8c3da1e5bcde2e9813107101f6c4bc45634b2d7f035969dba4c74ac
Oct 30 19:02:17.036676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.036216    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"]
Oct 30 19:02:17.048197 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.048166    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"]
Oct 30 19:02:17.048368 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.048328    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.055889 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.055867    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-c2e3f9-serving-cert\""
Oct 30 19:02:17.226266 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.205884    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee31e82e-3665-4a8c-b502-429976b05fd9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.226564 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.226534    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.226824 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.226807    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.227137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.227113    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbmvx\" (UniqueName: \"kubernetes.io/projected/ee31e82e-3665-4a8c-b502-429976b05fd9-kube-api-access-qbmvx\") pod \"maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.345516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.324341    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:17.376154 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.366309    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbmvx\" (UniqueName: \"kubernetes.io/projected/ee31e82e-3665-4a8c-b502-429976b05fd9-kube-api-access-qbmvx\") pod \"maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.412226 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.411969    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee31e82e-3665-4a8c-b502-429976b05fd9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.436646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.436608    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.437110 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.418972    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee31e82e-3665-4a8c-b502-429976b05fd9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.437296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.436458    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused" start-of-body=
Oct 30 19:02:17.437425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.437405    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused"
Oct 30 19:02:17.437639 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.437023    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.437944 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.431773    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbmvx\" (UniqueName: \"kubernetes.io/projected/ee31e82e-3665-4a8c-b502-429976b05fd9-kube-api-access-qbmvx\") pod \"maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.440774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.440434    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.441015 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.440994    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.442308 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.442189    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:17.442491 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.442470    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:02:17.442583 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.442510    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:02:17.553762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.553405    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:17.640122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.634310    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:17.664576 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.664540    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq" event={"ID":"d8afa108-8874-4821-9a23-784ff39e42ce","Type":"ContainerStarted","Data":"33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc"}
Oct 30 19:02:17.692891 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.692846    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"
Oct 30 19:02:17.693187 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.693155    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq" event={"ID":"d8afa108-8874-4821-9a23-784ff39e42ce","Type":"ContainerStarted","Data":"15f2689fe8c3da1e5bcde2e9813107101f6c4bc45634b2d7f035969dba4c74ac"}
Oct 30 19:02:17.742123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:17.720905    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq" podStartSLOduration=1.7208822929999998 podStartE2EDuration="1.720882293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:17.703973615 +0000 UTC m=+1559.420318692" watchObservedRunningTime="2026-10-30 19:02:17.720882293 +0000 UTC m=+1559.437227371"
Oct 30 19:02:18.222409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:18.222374    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:18.497130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:18.490661    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused" start-of-body=
Oct 30 19:02:18.497130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:18.490720    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused"
Oct 30 19:02:18.619153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:18.613014    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:19.018170 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:18.997686    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"]
Oct 30 19:02:19.033372 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.033336    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy" containerID="cri-o://b16e8765069999135852c2a9c8c1e26d9120c45e4d972f279b6a77fcd11b58f2" gracePeriod=30
Oct 30 19:02:19.045367 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.019332    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:02:19.046915 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.046870    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:02:19.048687 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.048660    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"]
Oct 30 19:02:19.091066 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.091032    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:19.282725 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.282586    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:19.433783 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.433749    4925 patch_prober.go:29] interesting pod/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:02:19.446273 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.446214    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:02:19.452122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.450565    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:02:19.454822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.450614    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:02:19.454963 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.454931    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused" start-of-body=
Oct 30 19:02:19.455041 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.454965    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused"
Oct 30 19:02:19.936785 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:19.936748    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg" event={"ID":"ee31e82e-3665-4a8c-b502-429976b05fd9","Type":"ContainerStarted","Data":"040d69b26471784271ce26b171510c3bf09c60177376a1424278626f437ef95c"}
Oct 30 19:02:20.216123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:20.213905    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:20.418172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:20.413585    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:20.436134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:20.436058    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused" start-of-body=
Oct 30 19:02:20.436319 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:20.436160    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused"
Oct 30 19:02:20.441433 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:20.441401    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:02:20.441584 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:20.441452    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:02:20.706838 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:20.706787    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:20.946795 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:20.946756    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg" event={"ID":"ee31e82e-3665-4a8c-b502-429976b05fd9","Type":"ContainerStarted","Data":"d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe"}
Oct 30 19:02:20.947039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:20.946845    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:21.253474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:21.253433    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"
Oct 30 19:02:21.256258 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:21.255394    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"
Oct 30 19:02:21.286858 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:21.286780    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg" podStartSLOduration=4.709247817 podStartE2EDuration="5.286760355s" totalImagesPullingTime="577.512538ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:20.970366054 +0000 UTC m=+1562.686711131" watchObservedRunningTime="2026-10-30 19:02:21.286760355 +0000 UTC m=+1563.003105432"
Oct 30 19:02:21.445238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:21.444531    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:02:21.445238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:21.444595    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:02:21.453578 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:21.452764    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused" start-of-body=
Oct 30 19:02:21.453578 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:21.452843    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused"
Oct 30 19:02:21.661915 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:21.661883    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:21.896247 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:21.881824    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:22.436395 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.436369    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused" start-of-body=
Oct 30 19:02:22.436673 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.436613    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused"
Oct 30 19:02:22.442291 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.442268    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:02:22.442462 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.442441    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:02:22.620623 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.620588    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:22.888185 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.885971    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:22.919117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.915752    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"]
Oct 30 19:02:22.943137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.933854    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"]
Oct 30 19:02:22.943137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.933964    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"
Oct 30 19:02:22.943137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.939121    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"]
Oct 30 19:02:22.947257 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.944745    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"
Oct 30 19:02:22.956146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.953626    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w4-ed012e\""
Oct 30 19:02:22.999308 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:22.951745    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w4-ed012e-dockercfg-j22q9\""
Oct 30 19:02:23.035491 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.034970    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"]
Oct 30 19:02:23.085292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.069795    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c79b524-09c4-4e29-8a1c-2306621221db-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl\" (UID: \"3c79b524-09c4-4e29-8a1c-2306621221db\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"
Oct 30 19:02:23.085292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.069859    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2640e252-a00f-462c-bd40-760ba7bc79e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv\" (UID: \"2640e252-a00f-462c-bd40-760ba7bc79e0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"
Oct 30 19:02:23.085292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.069906    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2smp9\" (UniqueName: \"kubernetes.io/projected/2640e252-a00f-462c-bd40-760ba7bc79e0-kube-api-access-2smp9\") pod \"payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv\" (UID: \"2640e252-a00f-462c-bd40-760ba7bc79e0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"
Oct 30 19:02:23.085292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.070032    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7brkp\" (UniqueName: \"kubernetes.io/projected/3c79b524-09c4-4e29-8a1c-2306621221db-kube-api-access-7brkp\") pod \"payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl\" (UID: \"3c79b524-09c4-4e29-8a1c-2306621221db\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"
Oct 30 19:02:23.130712 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.113930    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:23.223011 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.220875    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7brkp\" (UniqueName: \"kubernetes.io/projected/3c79b524-09c4-4e29-8a1c-2306621221db-kube-api-access-7brkp\") pod \"payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl\" (UID: \"3c79b524-09c4-4e29-8a1c-2306621221db\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"
Oct 30 19:02:23.223603 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.223546    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c79b524-09c4-4e29-8a1c-2306621221db-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl\" (UID: \"3c79b524-09c4-4e29-8a1c-2306621221db\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"
Oct 30 19:02:23.223817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.223782    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2640e252-a00f-462c-bd40-760ba7bc79e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv\" (UID: \"2640e252-a00f-462c-bd40-760ba7bc79e0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"
Oct 30 19:02:23.223956 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.223940    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2smp9\" (UniqueName: \"kubernetes.io/projected/2640e252-a00f-462c-bd40-760ba7bc79e0-kube-api-access-2smp9\") pod \"payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv\" (UID: \"2640e252-a00f-462c-bd40-760ba7bc79e0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"
Oct 30 19:02:23.226027 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.226001    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c79b524-09c4-4e29-8a1c-2306621221db-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl\" (UID: \"3c79b524-09c4-4e29-8a1c-2306621221db\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"
Oct 30 19:02:23.227021 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.227000    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2640e252-a00f-462c-bd40-760ba7bc79e0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv\" (UID: \"2640e252-a00f-462c-bd40-760ba7bc79e0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"
Oct 30 19:02:23.434143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.427411    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7brkp\" (UniqueName: \"kubernetes.io/projected/3c79b524-09c4-4e29-8a1c-2306621221db-kube-api-access-7brkp\") pod \"payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl\" (UID: \"3c79b524-09c4-4e29-8a1c-2306621221db\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"
Oct 30 19:02:23.434143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.429709    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2smp9\" (UniqueName: \"kubernetes.io/projected/2640e252-a00f-462c-bd40-760ba7bc79e0-kube-api-access-2smp9\") pod \"payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv\" (UID: \"2640e252-a00f-462c-bd40-760ba7bc79e0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"
Oct 30 19:02:23.436839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.436065    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused" start-of-body=
Oct 30 19:02:23.436839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.436163    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused"
Oct 30 19:02:23.442640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.442582    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:02:23.442853 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.442830    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:02:23.529123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.513629    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"
Oct 30 19:02:23.623723 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:23.600523    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"
Oct 30 19:02:24.181811 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.181342    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"]
Oct 30 19:02:24.185140 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:02:24.185055    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2640e252_a00f_462c_bd40_760ba7bc79e0.slice/crio-c7b845d17d867b3d8739600a3e551aa05768b6277566cff6c8af5b290cb45445 WatchSource:0}: Error finding container c7b845d17d867b3d8739600a3e551aa05768b6277566cff6c8af5b290cb45445: Status 404 returned error can't find the container with id c7b845d17d867b3d8739600a3e551aa05768b6277566cff6c8af5b290cb45445
Oct 30 19:02:24.201960 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.201934    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"]
Oct 30 19:02:24.223905 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:02:24.223863    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c79b524_09c4_4e29_8a1c_2306621221db.slice/crio-50afc861bcc2fcb40eba225b921a58d9893f4e91dd441ec52200db56e88bb91f WatchSource:0}: Error finding container 50afc861bcc2fcb40eba225b921a58d9893f4e91dd441ec52200db56e88bb91f: Status 404 returned error can't find the container with id 50afc861bcc2fcb40eba225b921a58d9893f4e91dd441ec52200db56e88bb91f
Oct 30 19:02:24.408800 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.395040    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:24.507115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.506420    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused" start-of-body=
Oct 30 19:02:24.507115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.506488    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.176:15021/healthz/ready\": dial tcp 10.132.0.176:15021: connect: connection refused"
Oct 30 19:02:24.507115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.506879    4925 patch_prober.go:29] interesting pod/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:02:24.507115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.506911    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:02:24.557051 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.556961    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv" event={"ID":"2640e252-a00f-462c-bd40-760ba7bc79e0","Type":"ContainerStarted","Data":"c7b845d17d867b3d8739600a3e551aa05768b6277566cff6c8af5b290cb45445"}
Oct 30 19:02:24.560378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.559996    4925 generic.go:356] "Generic (PLEG): container finished" podID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerID="b16e8765069999135852c2a9c8c1e26d9120c45e4d972f279b6a77fcd11b58f2" exitCode=0
Oct 30 19:02:24.560378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.560219    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" event={"ID":"824748a3-c26d-4699-a479-9a5be67f3c0d","Type":"ContainerDied","Data":"b16e8765069999135852c2a9c8c1e26d9120c45e4d972f279b6a77fcd11b58f2"}
Oct 30 19:02:24.560378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.560270    4925 scope.go:122] "RemoveContainer" containerID="fc13163f5124fd3b9c9b7b77d70ad5522ee0cc6640ee149c054b1529ec077c7c"
Oct 30 19:02:24.565054 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.564944    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl" event={"ID":"3c79b524-09c4-4e29-8a1c-2306621221db","Type":"ContainerStarted","Data":"50afc861bcc2fcb40eba225b921a58d9893f4e91dd441ec52200db56e88bb91f"}
Oct 30 19:02:24.637880 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.637851    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 19:02:24.808587 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.808509    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/824748a3-c26d-4699-a479-9a5be67f3c0d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/824748a3-c26d-4699-a479-9a5be67f3c0d-istiod-ca-cert\") pod \"824748a3-c26d-4699-a479-9a5be67f3c0d\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") "
Oct 30 19:02:24.808587 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.808569    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-credential-socket\") pod \"824748a3-c26d-4699-a479-9a5be67f3c0d\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") "
Oct 30 19:02:24.808863 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.808695    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-podinfo\") pod \"824748a3-c26d-4699-a479-9a5be67f3c0d\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") "
Oct 30 19:02:24.808863 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.808719    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-kube-api-access-lqm7q\" (UniqueName: \"kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-kube-api-access-lqm7q\") pod \"824748a3-c26d-4699-a479-9a5be67f3c0d\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") "
Oct 30 19:02:24.808863 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.808745    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-certs\") pod \"824748a3-c26d-4699-a479-9a5be67f3c0d\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") "
Oct 30 19:02:24.808863 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.808808    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-data\") pod \"824748a3-c26d-4699-a479-9a5be67f3c0d\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") "
Oct 30 19:02:24.809052 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.808894    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-credential-socket" pod "824748a3-c26d-4699-a479-9a5be67f3c0d" (UID: "824748a3-c26d-4699-a479-9a5be67f3c0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:24.809052 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.808939    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/824748a3-c26d-4699-a479-9a5be67f3c0d-istiod-ca-cert" pod "824748a3-c26d-4699-a479-9a5be67f3c0d" (UID: "824748a3-c26d-4699-a479-9a5be67f3c0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:24.809052 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.808955    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-token\" (UniqueName: \"kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-token\") pod \"824748a3-c26d-4699-a479-9a5be67f3c0d\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") "
Oct 30 19:02:24.809052 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.808986    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-socket\") pod \"824748a3-c26d-4699-a479-9a5be67f3c0d\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") "
Oct 30 19:02:24.809288 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.809056    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-envoy\") pod \"824748a3-c26d-4699-a479-9a5be67f3c0d\" (UID: \"824748a3-c26d-4699-a479-9a5be67f3c0d\") "
Oct 30 19:02:24.809359 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.809299    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-data" pod "824748a3-c26d-4699-a479-9a5be67f3c0d" (UID: "824748a3-c26d-4699-a479-9a5be67f3c0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:24.809359 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.809348    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-certs" pod "824748a3-c26d-4699-a479-9a5be67f3c0d" (UID: "824748a3-c26d-4699-a479-9a5be67f3c0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:24.809544 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.809482    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-socket" pod "824748a3-c26d-4699-a479-9a5be67f3c0d" (UID: "824748a3-c26d-4699-a479-9a5be67f3c0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:24.809802 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.809782    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/824748a3-c26d-4699-a479-9a5be67f3c0d-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:24.809802 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.809801    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:24.810001 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.809810    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:24.810001 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.809819    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:24.810001 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.809829    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:24.813020 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.811983    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-kube-api-access-lqm7q" pod "824748a3-c26d-4699-a479-9a5be67f3c0d" (UID: "824748a3-c26d-4699-a479-9a5be67f3c0d"). InnerVolumeSpecName "kube-api-access-lqm7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:24.813248 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.813226    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-envoy" pod "824748a3-c26d-4699-a479-9a5be67f3c0d" (UID: "824748a3-c26d-4699-a479-9a5be67f3c0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:24.813392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.813328    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-token" pod "824748a3-c26d-4699-a479-9a5be67f3c0d" (UID: "824748a3-c26d-4699-a479-9a5be67f3c0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:24.813496 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.813433    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-podinfo" pod "824748a3-c26d-4699-a479-9a5be67f3c0d" (UID: "824748a3-c26d-4699-a479-9a5be67f3c0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:02:24.911024 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.910989    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:24.911024 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.911018    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:24.911024 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.911029    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/824748a3-c26d-4699-a479-9a5be67f3c0d-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:24.911378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:24.911038    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqm7q\" (UniqueName: \"kubernetes.io/projected/824748a3-c26d-4699-a479-9a5be67f3c0d-kube-api-access-lqm7q\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:25.378583 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.378549    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:25.581269 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.581241    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"
Oct 30 19:02:25.581560 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.581269    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9" event={"ID":"824748a3-c26d-4699-a479-9a5be67f3c0d","Type":"ContainerDied","Data":"21cac42abce464833c9377b296cfea6fd0315cfa2d2f257f83809dd05dc7c29e"}
Oct 30 19:02:25.581650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.581602    4925 scope.go:122] "RemoveContainer" containerID="b16e8765069999135852c2a9c8c1e26d9120c45e4d972f279b6a77fcd11b58f2"
Oct 30 19:02:25.585130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.585061    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl" event={"ID":"3c79b524-09c4-4e29-8a1c-2306621221db","Type":"ContainerStarted","Data":"2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af"}
Oct 30 19:02:25.586120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.586061    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"
Oct 30 19:02:25.591755 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.591727    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv" event={"ID":"2640e252-a00f-462c-bd40-760ba7bc79e0","Type":"ContainerStarted","Data":"52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d"}
Oct 30 19:02:25.593377 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.592582    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"
Oct 30 19:02:25.600678 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.599165    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:25.601502 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.601161    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:25.606556 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.606505    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:02:25.610604 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.610581    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:25.612455 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.611766    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:25.612455 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.611841    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl" podStartSLOduration=3.6118274660000003 podStartE2EDuration="3.611827466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:25.605472654 +0000 UTC m=+1567.321817731" watchObservedRunningTime="2026-10-30 19:02:25.611827466 +0000 UTC m=+1567.328172543"
Oct 30 19:02:25.613276 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.613256    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:02:25.646456 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.646347    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv" podStartSLOduration=3.646331639 podStartE2EDuration="3.646331639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:25.643554608 +0000 UTC m=+1567.359899697" watchObservedRunningTime="2026-10-30 19:02:25.646331639 +0000 UTC m=+1567.362676714"
Oct 30 19:02:25.667184 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.667155    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"]
Oct 30 19:02:25.668825 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:25.668712    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-6a327a-openshift-default-958795d8f-hn2q9"]
Oct 30 19:02:26.147881 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.146789    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:26.423851 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.423819    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" path="/var/lib/kubelet/pods/824748a3-c26d-4699-a479-9a5be67f3c0d/volumes"
Oct 30 19:02:26.729272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.729164    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"]
Oct 30 19:02:26.730246 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.730220    4925 cpu_manager.go:414] "containerMap: removing container" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy"
Oct 30 19:02:26.730451 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.730425    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy"
Oct 30 19:02:26.730547 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.730473    4925 cpu_manager.go:414] "containerMap: removing container" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy"
Oct 30 19:02:26.730547 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.730482    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy"
Oct 30 19:02:26.730714 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.730699    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy"
Oct 30 19:02:26.730773 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.730722    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="824748a3-c26d-4699-a479-9a5be67f3c0d" containerName="istio-proxy"
Oct 30 19:02:26.736228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.736202    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:26.742876 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.742848    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"]
Oct 30 19:02:26.744398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.744351    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-baa504-serving-cert\""
Oct 30 19:02:26.859361 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.859316    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:26.876782 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.876755    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:26.876927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.876829    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:26.877077 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.877050    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xmq8\" (UniqueName: \"kubernetes.io/projected/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-kube-api-access-2xmq8\") pod \"maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:26.877173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.877130    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:26.987411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.982322    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:26.987411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.982468    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:26.987411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.982652    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xmq8\" (UniqueName: \"kubernetes.io/projected/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-kube-api-access-2xmq8\") pod \"maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:26.987411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.982705    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:26.987411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:26.983962    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:27.002118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:27.002059    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:27.009119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:27.006737    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xmq8\" (UniqueName: \"kubernetes.io/projected/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-kube-api-access-2xmq8\") pod \"maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:27.012968 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:27.010669    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:27.046608 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:27.046438    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:27.085789 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:27.085760    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:27.113991 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:27.113964    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:02:27.506303 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:27.500845    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"]
Oct 30 19:02:27.612175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:27.612143    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc" event={"ID":"778ba6a8-bfbe-4e63-ae97-bff72ebfb931","Type":"ContainerStarted","Data":"5ea34228c15facc7152293f211ced0f2e2d2fa137bafe1df2dd423ac06823fd7"}
Oct 30 19:02:27.885200 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:27.884762    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:28.573205 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:28.569294    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:28.635338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:28.635304    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc" event={"ID":"778ba6a8-bfbe-4e63-ae97-bff72ebfb931","Type":"ContainerStarted","Data":"ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7"}
Oct 30 19:02:28.635516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:28.635364    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:28.653869 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:28.653823    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc" podStartSLOduration=2.160155609 podStartE2EDuration="2.6538069s" totalImagesPullingTime="493.651291ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:28.652447086 +0000 UTC m=+1570.368792163" watchObservedRunningTime="2026-10-30 19:02:28.6538069 +0000 UTC m=+1570.370151977"
Oct 30 19:02:29.292120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:29.285820    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:29.940432 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:29.940407    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"
Oct 30 19:02:29.946041 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:29.946009    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:31.112122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:31.109290    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:31.295142 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:31.292825    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:31.651121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:31.644039    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:33.396136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.385188    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:33.405124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.402191    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:33.609125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.608209    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:02:33.813461 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.813366    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"]
Oct 30 19:02:33.829707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.829675    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"]
Oct 30 19:02:33.833120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.829972    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:33.842970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.842939    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-ed012e-serving-cert\""
Oct 30 19:02:33.914167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.911591    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:33.914167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.911731    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhf5q\" (UniqueName: \"kubernetes.io/projected/0a2211bc-9633-49ee-b65e-7c39f753357d-kube-api-access-lhf5q\") pod \"maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:33.914167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.911890    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2211bc-9633-49ee-b65e-7c39f753357d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:33.914167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.911967    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:33.920026 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.919978    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"]
Oct 30 19:02:33.924683 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:33.924655    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"]
Oct 30 19:02:34.015413 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:34.015376    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:34.015734 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:34.015710    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhf5q\" (UniqueName: \"kubernetes.io/projected/0a2211bc-9633-49ee-b65e-7c39f753357d-kube-api-access-lhf5q\") pod \"maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:34.015969 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:34.015950    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2211bc-9633-49ee-b65e-7c39f753357d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:34.016134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:34.016117    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:34.017004 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:34.016978    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2211bc-9633-49ee-b65e-7c39f753357d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:34.027173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:34.027144    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhf5q\" (UniqueName: \"kubernetes.io/projected/0a2211bc-9633-49ee-b65e-7c39f753357d-kube-api-access-lhf5q\") pod \"maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:34.028075 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:34.028051    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:34.029382 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:34.029360    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:34.175168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:34.173890    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:34.843132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:34.843069    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"]
Oct 30 19:02:34.843569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:34.843205    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:02:34.914687 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:34.914646    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr" event={"ID":"0a2211bc-9633-49ee-b65e-7c39f753357d","Type":"ContainerStarted","Data":"deee0b2851747e3b51b81b48ae294fabe379149b36775232c3a45b9a4f9b8742"}
Oct 30 19:02:35.932417 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:35.932385    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr" event={"ID":"0a2211bc-9633-49ee-b65e-7c39f753357d","Type":"ContainerStarted","Data":"05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029"}
Oct 30 19:02:35.932992 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:35.932975    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:35.953131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:35.953065    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr" podStartSLOduration=2.379522029 podStartE2EDuration="2.953049466s" totalImagesPullingTime="573.527437ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:35.950646813 +0000 UTC m=+1577.666991892" watchObservedRunningTime="2026-10-30 19:02:35.953049466 +0000 UTC m=+1577.669394543"
Oct 30 19:02:36.000542 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:36.000510    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:36.198952 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:36.198817    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:36.582256 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:36.582119    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:37.125657 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:37.125620    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:37.302437 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:37.302398    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:37.615883 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:37.615854    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"
Oct 30 19:02:37.616066 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:37.615945    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"
Oct 30 19:02:37.896866 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:37.896776    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:38.710263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:38.710166    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:38.921405 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:38.921353    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9" podUID="fc42e32a-caae-45ee-a16c-bf281e087e1b" containerName="payload-processing" containerID="cri-o://b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367" gracePeriod=25
Oct 30 19:02:38.925471 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:38.925443    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk" podUID="dd91f256-8f26-4ed2-97c7-c2835bb0f90a" containerName="payload-pre-processing" containerID="cri-o://a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e" gracePeriod=25
Oct 30 19:02:39.311661 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.311628    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"
Oct 30 19:02:39.368923 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.368896    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"
Oct 30 19:02:39.449390 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.449356    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc42e32a-caae-45ee-a16c-bf281e087e1b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc42e32a-caae-45ee-a16c-bf281e087e1b-plugins-config-volume\") pod \"fc42e32a-caae-45ee-a16c-bf281e087e1b\" (UID: \"fc42e32a-caae-45ee-a16c-bf281e087e1b\") "
Oct 30 19:02:39.449578 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.449555    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc42e32a-caae-45ee-a16c-bf281e087e1b-kube-api-access-mbnwv\" (UniqueName: \"kubernetes.io/projected/fc42e32a-caae-45ee-a16c-bf281e087e1b-kube-api-access-mbnwv\") pod \"fc42e32a-caae-45ee-a16c-bf281e087e1b\" (UID: \"fc42e32a-caae-45ee-a16c-bf281e087e1b\") "
Oct 30 19:02:39.449701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.449681    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-plugins-config-volume\") pod \"dd91f256-8f26-4ed2-97c7-c2835bb0f90a\" (UID: \"dd91f256-8f26-4ed2-97c7-c2835bb0f90a\") "
Oct 30 19:02:39.449787 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.449760    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc42e32a-caae-45ee-a16c-bf281e087e1b-plugins-config-volume" pod "fc42e32a-caae-45ee-a16c-bf281e087e1b" (UID: "fc42e32a-caae-45ee-a16c-bf281e087e1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:39.450064 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.450045    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-plugins-config-volume" pod "dd91f256-8f26-4ed2-97c7-c2835bb0f90a" (UID: "dd91f256-8f26-4ed2-97c7-c2835bb0f90a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:39.450210 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.450194    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-kube-api-access-295t7\" (UniqueName: \"kubernetes.io/projected/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-kube-api-access-295t7\") pod \"dd91f256-8f26-4ed2-97c7-c2835bb0f90a\" (UID: \"dd91f256-8f26-4ed2-97c7-c2835bb0f90a\") "
Oct 30 19:02:39.450708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.450690    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:39.450708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.450708    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc42e32a-caae-45ee-a16c-bf281e087e1b-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:39.452203 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.452175    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc42e32a-caae-45ee-a16c-bf281e087e1b-kube-api-access-mbnwv" pod "fc42e32a-caae-45ee-a16c-bf281e087e1b" (UID: "fc42e32a-caae-45ee-a16c-bf281e087e1b"). InnerVolumeSpecName "kube-api-access-mbnwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:39.452778 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.452761    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-kube-api-access-295t7" pod "dd91f256-8f26-4ed2-97c7-c2835bb0f90a" (UID: "dd91f256-8f26-4ed2-97c7-c2835bb0f90a"). InnerVolumeSpecName "kube-api-access-295t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:39.541217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.534163    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:39.557121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.555548    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-295t7\" (UniqueName: \"kubernetes.io/projected/dd91f256-8f26-4ed2-97c7-c2835bb0f90a-kube-api-access-295t7\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:39.557121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.555588    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbnwv\" (UniqueName: \"kubernetes.io/projected/fc42e32a-caae-45ee-a16c-bf281e087e1b-kube-api-access-mbnwv\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:02:39.993622 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.993585    4925 generic.go:356] "Generic (PLEG): container finished" podID="dd91f256-8f26-4ed2-97c7-c2835bb0f90a" containerID="a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e" exitCode=0
Oct 30 19:02:39.994121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.993643    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"
Oct 30 19:02:39.994121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.993702    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk" event={"ID":"dd91f256-8f26-4ed2-97c7-c2835bb0f90a","Type":"ContainerDied","Data":"a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e"}
Oct 30 19:02:39.994121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.993751    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk" event={"ID":"dd91f256-8f26-4ed2-97c7-c2835bb0f90a","Type":"ContainerDied","Data":"182d464a94310fa6cab3ce893f6c24ac69511673742ee367f5d1019dac956254"}
Oct 30 19:02:39.994121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:39.993780    4925 scope.go:122] "RemoveContainer" containerID="a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e"
Oct 30 19:02:40.000885 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.000859    4925 generic.go:356] "Generic (PLEG): container finished" podID="fc42e32a-caae-45ee-a16c-bf281e087e1b" containerID="b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367" exitCode=0
Oct 30 19:02:40.001015 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.000913    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"
Oct 30 19:02:40.001214 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.001167    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9" event={"ID":"fc42e32a-caae-45ee-a16c-bf281e087e1b","Type":"ContainerDied","Data":"b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367"}
Oct 30 19:02:40.001617 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.001593    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9" event={"ID":"fc42e32a-caae-45ee-a16c-bf281e087e1b","Type":"ContainerDied","Data":"5f8a3636d496c5bc3e7017623b5400aafd201bc053efab12ce853b47c7b985a9"}
Oct 30 19:02:40.014770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.014750    4925 scope.go:122] "RemoveContainer" containerID="a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e"
Oct 30 19:02:40.015071 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:02:40.015045    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e\": container with ID starting with a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e not found: ID does not exist" containerID="a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e"
Oct 30 19:02:40.015187 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.015080    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e"} err="failed to get container status \"a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e\": rpc error: code = NotFound desc = could not find container \"a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e\": container with ID starting with a562ba84d0d569ca5829d559bb65a89e0fb233189631f49ee519f5fddb7a955e not found: ID does not exist"
Oct 30 19:02:40.015187 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.015146    4925 scope.go:122] "RemoveContainer" containerID="b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367"
Oct 30 19:02:40.020556 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.020539    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"]
Oct 30 19:02:40.027499 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.027473    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-a014d1-77457c8579-jlhrk"]
Oct 30 19:02:40.032238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.032219    4925 scope.go:122] "RemoveContainer" containerID="b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367"
Oct 30 19:02:40.032541 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:02:40.032523    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367\": container with ID starting with b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367 not found: ID does not exist" containerID="b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367"
Oct 30 19:02:40.032585 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.032550    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367"} err="failed to get container status \"b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367\": rpc error: code = NotFound desc = could not find container \"b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367\": container with ID starting with b870701691a8b8a3899189816836fe1d114e26a8fca5d1027188b702cf779367 not found: ID does not exist"
Oct 30 19:02:40.040719 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.040549    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"]
Oct 30 19:02:40.045356 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.045338    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-a014d1-6c8d56cb-wxpj9"]
Oct 30 19:02:40.427323 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.427294    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd91f256-8f26-4ed2-97c7-c2835bb0f90a" path="/var/lib/kubelet/pods/dd91f256-8f26-4ed2-97c7-c2835bb0f90a/volumes"
Oct 30 19:02:40.427710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.427697    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc42e32a-caae-45ee-a16c-bf281e087e1b" path="/var/lib/kubelet/pods/fc42e32a-caae-45ee-a16c-bf281e087e1b/volumes"
Oct 30 19:02:40.504809 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.504768    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:40.717173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:40.713083    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:41.875021 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:41.874962    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:41.950774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:41.950664    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:02:42.232632 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.232598    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"]
Oct 30 19:02:42.233768 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.233733    4925 cpu_manager.go:414] "containerMap: removing container" podUID="dd91f256-8f26-4ed2-97c7-c2835bb0f90a" containerName="payload-pre-processing"
Oct 30 19:02:42.233768 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.233762    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd91f256-8f26-4ed2-97c7-c2835bb0f90a" containerName="payload-pre-processing"
Oct 30 19:02:42.233953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.233864    4925 cpu_manager.go:414] "containerMap: removing container" podUID="fc42e32a-caae-45ee-a16c-bf281e087e1b" containerName="payload-processing"
Oct 30 19:02:42.233953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.233874    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc42e32a-caae-45ee-a16c-bf281e087e1b" containerName="payload-processing"
Oct 30 19:02:42.234152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.234121    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd91f256-8f26-4ed2-97c7-c2835bb0f90a" containerName="payload-pre-processing"
Oct 30 19:02:42.234152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.234150    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc42e32a-caae-45ee-a16c-bf281e087e1b" containerName="payload-processing"
Oct 30 19:02:42.240903 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.240137    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"
Oct 30 19:02:42.244306 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.244266    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"]
Oct 30 19:02:42.245958 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.245908    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:42.246301 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.246278    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-2d1c71-dockercfg-7tlhj\""
Oct 30 19:02:42.251667 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.251628    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-2d1c71\""
Oct 30 19:02:42.339129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.325427    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c71c1a5f-6d34-4b5b-854c-ae1577419337-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf\" (UID: \"c71c1a5f-6d34-4b5b-854c-ae1577419337\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"
Oct 30 19:02:42.339129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.325509    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hn9q\" (UniqueName: \"kubernetes.io/projected/c71c1a5f-6d34-4b5b-854c-ae1577419337-kube-api-access-6hn9q\") pod \"payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf\" (UID: \"c71c1a5f-6d34-4b5b-854c-ae1577419337\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"
Oct 30 19:02:42.452265 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.452215    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c71c1a5f-6d34-4b5b-854c-ae1577419337-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf\" (UID: \"c71c1a5f-6d34-4b5b-854c-ae1577419337\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"
Oct 30 19:02:42.452472 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.452293    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hn9q\" (UniqueName: \"kubernetes.io/projected/c71c1a5f-6d34-4b5b-854c-ae1577419337-kube-api-access-6hn9q\") pod \"payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf\" (UID: \"c71c1a5f-6d34-4b5b-854c-ae1577419337\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"
Oct 30 19:02:42.453589 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.453555    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c71c1a5f-6d34-4b5b-854c-ae1577419337-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf\" (UID: \"c71c1a5f-6d34-4b5b-854c-ae1577419337\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"
Oct 30 19:02:42.492291 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.475205    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:42.505180 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.501740    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hn9q\" (UniqueName: \"kubernetes.io/projected/c71c1a5f-6d34-4b5b-854c-ae1577419337-kube-api-access-6hn9q\") pod \"payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf\" (UID: \"c71c1a5f-6d34-4b5b-854c-ae1577419337\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"
Oct 30 19:02:42.599131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.593649    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"
Oct 30 19:02:42.943897 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:02:42.943865    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc71c1a5f_6d34_4b5b_854c_ae1577419337.slice/crio-b799175680963112ffc359386d2add468f3529c4bebe60bb4380321e1b32864e WatchSource:0}: Error finding container b799175680963112ffc359386d2add468f3529c4bebe60bb4380321e1b32864e: Status 404 returned error can't find the container with id b799175680963112ffc359386d2add468f3529c4bebe60bb4380321e1b32864e
Oct 30 19:02:42.944371 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:42.944038    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"]
Oct 30 19:02:43.039893 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:43.010369    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:43.215887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:43.215735    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:43.218651 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:43.217924    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf" event={"ID":"c71c1a5f-6d34-4b5b-854c-ae1577419337","Type":"ContainerStarted","Data":"b799175680963112ffc359386d2add468f3529c4bebe60bb4380321e1b32864e"}
Oct 30 19:02:43.315022 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:02:43.314987    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:02:43.435119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:43.434821    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:43.650176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:43.649183    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:44.228029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:44.227998    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf" event={"ID":"c71c1a5f-6d34-4b5b-854c-ae1577419337","Type":"ContainerStarted","Data":"1025e4ef46b2956f0245f984392a9f3f892baf4869f90527f0dc58e9b37d8a7a"}
Oct 30 19:02:44.228471 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:44.228107    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"
Oct 30 19:02:44.246486 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:44.246436    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf" podStartSLOduration=2.24642412 podStartE2EDuration="2.24642412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:44.244051078 +0000 UTC m=+1585.960396155" watchObservedRunningTime="2026-10-30 19:02:44.24642412 +0000 UTC m=+1585.962769196"
Oct 30 19:02:44.552225 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:44.547308    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:44.833520 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:44.833438    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:45.290960 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:45.285871    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:45.304393 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:45.300527    4925 generic.go:356] "Generic (PLEG): container finished" podID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerID="45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0" exitCode=0
Oct 30 19:02:45.304393 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:45.301613    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" event={"ID":"ae0e1c00-9ab0-4e0b-911b-31e1843d9232","Type":"ContainerDied","Data":"45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0"}
Oct 30 19:02:45.304393 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:45.302054    4925 scope.go:122] "RemoveContainer" containerID="45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0"
Oct 30 19:02:45.305140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:45.304442    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:45.305140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:45.304525    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:45.305140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:45.304571    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:45.469115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:45.468509    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:45.684122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:45.677068    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:45.859492 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:45.859451    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:46.322568 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:46.322530    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" event={"ID":"ae0e1c00-9ab0-4e0b-911b-31e1843d9232","Type":"ContainerStarted","Data":"5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed"}
Oct 30 19:02:46.335750 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:46.335723    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:02:46.335936 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:46.335916    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:02:46.335983 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:46.335964    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 19:02:46.425181 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:46.403722    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:47.335946 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:47.335911    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:02:47.336392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:47.335960    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 19:02:47.459121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:47.459005    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:48.336133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:48.336083    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:02:48.336666 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:48.336236    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 19:02:48.639184 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:48.638805    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:49.336127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:49.335808    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:02:49.336127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:49.335871    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 19:02:49.408121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:49.406722    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:49.624130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:49.624014    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:49.809758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:49.809728    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:50.354246 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:50.346262    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:02:50.354246 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:50.346418    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 19:02:50.371122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:50.359675    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:50.846956 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:50.846917    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"]
Oct 30 19:02:50.853191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:50.853167    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"
Oct 30 19:02:50.886190 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:50.876654    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w0-68b466-dockercfg-947dc\""
Oct 30 19:02:50.886190 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:50.885962    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:50.901283 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:50.900982    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"]
Oct 30 19:02:50.916587 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:50.916551    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8kf9\" (UniqueName: \"kubernetes.io/projected/a98e91b2-25b6-4853-832f-e912c92ea068-kube-api-access-f8kf9\") pod \"payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn\" (UID: \"a98e91b2-25b6-4853-832f-e912c92ea068\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"
Oct 30 19:02:50.916849 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:50.916831    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a98e91b2-25b6-4853-832f-e912c92ea068-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn\" (UID: \"a98e91b2-25b6-4853-832f-e912c92ea068\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"
Oct 30 19:02:50.922317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:50.922292    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w0-68b466\""
Oct 30 19:02:51.039120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:51.031925    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8kf9\" (UniqueName: \"kubernetes.io/projected/a98e91b2-25b6-4853-832f-e912c92ea068-kube-api-access-f8kf9\") pod \"payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn\" (UID: \"a98e91b2-25b6-4853-832f-e912c92ea068\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"
Oct 30 19:02:51.039120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:51.032145    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a98e91b2-25b6-4853-832f-e912c92ea068-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn\" (UID: \"a98e91b2-25b6-4853-832f-e912c92ea068\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"
Oct 30 19:02:51.117175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:51.033085    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a98e91b2-25b6-4853-832f-e912c92ea068-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn\" (UID: \"a98e91b2-25b6-4853-832f-e912c92ea068\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"
Oct 30 19:02:51.219826 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:51.209855    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8kf9\" (UniqueName: \"kubernetes.io/projected/a98e91b2-25b6-4853-832f-e912c92ea068-kube-api-access-f8kf9\") pod \"payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn\" (UID: \"a98e91b2-25b6-4853-832f-e912c92ea068\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"
Oct 30 19:02:51.219826 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:51.211930    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:51.219826 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:51.212546    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"
Oct 30 19:02:51.351025 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:51.344433    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:02:51.351025 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:51.344496    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 19:02:51.389186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:51.387550    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:51.803886 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:51.803853    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:51.822228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:51.822206    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"]
Oct 30 19:02:52.451654 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:52.451613    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn" event={"ID":"a98e91b2-25b6-4853-832f-e912c92ea068","Type":"ContainerStarted","Data":"67229149ece0547c92f551084f91a7c81b2b9a5cf4811701b75cf2aa8a2ad1c7"}
Oct 30 19:02:52.452205 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:52.451671    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn" event={"ID":"a98e91b2-25b6-4853-832f-e912c92ea068","Type":"ContainerStarted","Data":"fafbd698f7896b7176644bbe3afa4133c2eb67710401fc374a551c684e7d87ac"}
Oct 30 19:02:52.452205 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:52.451720    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"
Oct 30 19:02:52.466637 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:52.466463    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:02:52.467159 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:52.467133    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:02:52.468343 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:52.468324    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:02:52.516235 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:52.516185    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn" podStartSLOduration=2.5161695010000003 podStartE2EDuration="2.516169501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:52.486849667 +0000 UTC m=+1594.203194744" watchObservedRunningTime="2026-10-30 19:02:52.516169501 +0000 UTC m=+1594.232514577"
Oct 30 19:02:52.618368 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:52.618322    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:52.845048 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:52.844959    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:53.038267 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:53.038230    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:53.590859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:53.590820    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:54.406121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:54.394722    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:54.604120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:54.603745    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:55.182121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:55.181649    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:55.305246 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:55.305215    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"
Oct 30 19:02:56.087982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:56.085898    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:57.274124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:57.274007    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:58.085051 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:58.085009    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:58.724126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:58.720773    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:59.331673 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:59.331638    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:59.923683 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:59.923649    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"]
Oct 30 19:02:59.937323 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:59.937296    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"]
Oct 30 19:02:59.938549 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:59.937959    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:02:59.945977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:02:59.945924    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-68b466-serving-cert\""
Oct 30 19:03:00.022291 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.022265    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.022456 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.022434    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b55892d-8b94-4401-bc08-65bd855471f5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.022579 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.022562    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.022646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.022633    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n677c\" (UniqueName: \"kubernetes.io/projected/3b55892d-8b94-4401-bc08-65bd855471f5-kube-api-access-n677c\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.108125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.101915    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:00.129128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.126401    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.129128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.126537    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n677c\" (UniqueName: \"kubernetes.io/projected/3b55892d-8b94-4401-bc08-65bd855471f5-kube-api-access-n677c\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.129128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.126655    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.129128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.126812    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b55892d-8b94-4401-bc08-65bd855471f5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.129128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.127737    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b55892d-8b94-4401-bc08-65bd855471f5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.151128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.136702    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.151128 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:03:00.136827    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-serving-cert: secret "maas-api-e2e-shared-b-w0-68b466-serving-cert" not found
Oct 30 19:03:00.151128 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:03:00.136905    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-maas-api-tls podName:3b55892d-8b94-4401-bc08-65bd855471f5 nodeName:}" failed. No retries permitted until 2026-10-30 19:03:00.636885209 +0000 UTC m=+1602.353230267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-maas-api-tls") pod "maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4" (UID: "3b55892d-8b94-4401-bc08-65bd855471f5") : secret "maas-api-e2e-shared-b-w0-68b466-serving-cert" not found
Oct 30 19:03:00.194170 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.178735    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n677c\" (UniqueName: \"kubernetes.io/projected/3b55892d-8b94-4401-bc08-65bd855471f5-kube-api-access-n677c\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.286683 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.286640    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:00.454571 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.454488    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:00.712218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.709379    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.736141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.731393    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:00.860410 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:00.860373    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:01.041203 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:01.035679    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:01.154565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:01.154510    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"]
Oct 30 19:03:01.155049 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:03:01.155026    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b55892d_8b94_4401_bc08_65bd855471f5.slice/crio-5fd55842d9ab70ea4f555492db414013e041f221300ad0dd82270c6f10db34f3 WatchSource:0}: Error finding container 5fd55842d9ab70ea4f555492db414013e041f221300ad0dd82270c6f10db34f3: Status 404 returned error can't find the container with id 5fd55842d9ab70ea4f555492db414013e041f221300ad0dd82270c6f10db34f3
Oct 30 19:03:01.615705 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:01.615670    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4" event={"ID":"3b55892d-8b94-4401-bc08-65bd855471f5","Type":"ContainerStarted","Data":"5fd55842d9ab70ea4f555492db414013e041f221300ad0dd82270c6f10db34f3"}
Oct 30 19:03:01.690399 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:01.690330    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:02.375131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:02.372153    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:02.626381 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:02.625985    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4" event={"ID":"3b55892d-8b94-4401-bc08-65bd855471f5","Type":"ContainerStarted","Data":"436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a"}
Oct 30 19:03:02.626381 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:02.626057    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:02.650400 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:02.650356    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4" podStartSLOduration=3.170661215 podStartE2EDuration="3.650341391s" totalImagesPullingTime="479.680176ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:02.647756321 +0000 UTC m=+1604.364101397" watchObservedRunningTime="2026-10-30 19:03:02.650341391 +0000 UTC m=+1604.366686466"
Oct 30 19:03:03.178974 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:03.173792    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:03.462151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:03.462057    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"
Oct 30 19:03:03.921497 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:03.921456    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:04.871120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:04.866679    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:05.162144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:05.162074    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:05.602177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:05.594642    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:05.814120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:05.804375    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:06.073518 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:06.073486    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:06.396663 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:06.396559    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:07.064133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:07.064048    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:07.245325 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:07.245288    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:07.848015 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:07.847981    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:08.040548 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:08.040503    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:08.640357 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:08.640329    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:03:09.025171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:09.025128    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:09.424371 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:09.424340    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:09.618757 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:09.618729    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:09.990900 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:09.987538    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:10.194837 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.194803    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:10.557707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.557676    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"]
Oct 30 19:03:10.562695 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.562674    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"
Oct 30 19:03:10.573655 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.573632    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-e9ada0\""
Oct 30 19:03:10.573801 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.573786    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-e9ada0-dockercfg-whzgm\""
Oct 30 19:03:10.576012 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.575989    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"]
Oct 30 19:03:10.607653 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.607622    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:10.741125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.717255    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dz7g\" (UniqueName: \"kubernetes.io/projected/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-kube-api-access-2dz7g\") pod \"payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh\" (UID: \"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4\") " pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"
Oct 30 19:03:10.764122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.757330    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-plugins-config-volume\") pod \"payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh\" (UID: \"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4\") " pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"
Oct 30 19:03:10.824168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.818353    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:10.870124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.866796    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dz7g\" (UniqueName: \"kubernetes.io/projected/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-kube-api-access-2dz7g\") pod \"payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh\" (UID: \"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4\") " pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"
Oct 30 19:03:10.870124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.866901    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-plugins-config-volume\") pod \"payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh\" (UID: \"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4\") " pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"
Oct 30 19:03:10.870124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.867995    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-plugins-config-volume\") pod \"payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh\" (UID: \"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4\") " pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"
Oct 30 19:03:10.925835 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:10.925800    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dz7g\" (UniqueName: \"kubernetes.io/projected/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-kube-api-access-2dz7g\") pod \"payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh\" (UID: \"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4\") " pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"
Oct 30 19:03:11.188881 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:11.188844    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"
Oct 30 19:03:11.472230 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:11.472203    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"]
Oct 30 19:03:11.473367 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:03:11.473327    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5f5b5b_3787_4149_84dd_fcc0a5f9b6a4.slice/crio-a81616cc6b6d3811820a4b5c8d02208ef7ab3a6aca844d10c22191597994d87d WatchSource:0}: Error finding container a81616cc6b6d3811820a4b5c8d02208ef7ab3a6aca844d10c22191597994d87d: Status 404 returned error can't find the container with id a81616cc6b6d3811820a4b5c8d02208ef7ab3a6aca844d10c22191597994d87d
Oct 30 19:03:11.556345 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:11.546771    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:11.892166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:11.892028    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh" event={"ID":"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4","Type":"ContainerStarted","Data":"ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926"}
Oct 30 19:03:11.892166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:11.892129    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh" event={"ID":"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4","Type":"ContainerStarted","Data":"a81616cc6b6d3811820a4b5c8d02208ef7ab3a6aca844d10c22191597994d87d"}
Oct 30 19:03:11.892166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:11.892168    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"
Oct 30 19:03:11.910614 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:11.910570    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh" podStartSLOduration=1.91055541 podStartE2EDuration="1.91055541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:11.909279771 +0000 UTC m=+1613.625624850" watchObservedRunningTime="2026-10-30 19:03:11.91055541 +0000 UTC m=+1613.626900487"
Oct 30 19:03:12.627067 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:12.627034    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:13.306445 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:03:13.306407    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:03:13.656871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:13.656712    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:14.804220 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:14.804139    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:14.992131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:14.988758    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:15.858125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:15.857631    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:16.812744 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:16.812707    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:19.336490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.336458    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:19.613001 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.612919    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"]
Oct 30 19:03:19.618955 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.618933    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.626970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.626941    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"]
Oct 30 19:03:19.629880 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.629852    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w4-ed012e\"/\"kube-root-ca.crt\""
Oct 30 19:03:19.630003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.629940    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w4-ed012e\"/\"default-dockercfg-fzhsr\""
Oct 30 19:03:19.630003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.629978    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w4-ed012e\"/\"auth-test-model-ed012e-kserve-self-signed-certs\""
Oct 30 19:03:19.630246 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.630228    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w4-ed012e\"/\"openshift-service-ca.crt\""
Oct 30 19:03:19.737642 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.737611    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:19.812109 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.803116    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-model-cache\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.832397 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.832366    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-tmp-dir\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.832908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.832879    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txfrc\" (UniqueName: \"kubernetes.io/projected/b920c3b4-6749-4ae3-895f-5de002df2777-kube-api-access-txfrc\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.833330 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.833308    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-home\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.833656 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.833639    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b920c3b4-6749-4ae3-895f-5de002df2777-tls-certs\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.833865 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.833850    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-dshm\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.948196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.947707    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-home\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.948196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.947834    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b920c3b4-6749-4ae3-895f-5de002df2777-tls-certs\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.948196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.947863    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-dshm\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.948196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.947982    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-model-cache\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.948196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.948010    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-tmp-dir\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.948196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.948139    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txfrc\" (UniqueName: \"kubernetes.io/projected/b920c3b4-6749-4ae3-895f-5de002df2777-kube-api-access-txfrc\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.949554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.949008    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-home\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.955133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.949931    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-model-cache\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:19.955133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:19.950557    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-tmp-dir\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:20.009133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:20.005807    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b920c3b4-6749-4ae3-895f-5de002df2777-tls-certs\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:20.034131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:20.031848    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txfrc\" (UniqueName: \"kubernetes.io/projected/b920c3b4-6749-4ae3-895f-5de002df2777-kube-api-access-txfrc\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:20.034131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:20.032629    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-dshm\") pod \"auth-test-model-ed012e-kserve-568d54c5c-k74vb\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") " pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:20.245798 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:20.245703    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:20.354001 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:20.353963    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:20.655744 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:03:20.655715    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb920c3b4_6749_4ae3_895f_5de002df2777.slice/crio-62fe2151cdb9bb393de41d86501cf281b107f1fe57ebf26f28901a9e0841d567 WatchSource:0}: Error finding container 62fe2151cdb9bb393de41d86501cf281b107f1fe57ebf26f28901a9e0841d567: Status 404 returned error can't find the container with id 62fe2151cdb9bb393de41d86501cf281b107f1fe57ebf26f28901a9e0841d567
Oct 30 19:03:20.656938 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:20.656904    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"]
Oct 30 19:03:21.098599 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:21.093798    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:21.106033 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:21.105992    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb" event={"ID":"b920c3b4-6749-4ae3-895f-5de002df2777","Type":"ContainerStarted","Data":"420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a"}
Oct 30 19:03:21.106422 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:21.106399    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb" event={"ID":"b920c3b4-6749-4ae3-895f-5de002df2777","Type":"ContainerStarted","Data":"62fe2151cdb9bb393de41d86501cf281b107f1fe57ebf26f28901a9e0841d567"}
Oct 30 19:03:21.128425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:21.128363    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb" podStartSLOduration=2.128345414 podStartE2EDuration="2.128345414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:21.126903729 +0000 UTC m=+1622.843248806" watchObservedRunningTime="2026-10-30 19:03:21.128345414 +0000 UTC m=+1622.844690492"
Oct 30 19:03:21.258309 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:21.257298    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:21.812697 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:21.800567    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:21.993882 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:21.993829    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:22.181305 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.180774    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"]
Oct 30 19:03:22.192082 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.187454    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.194852 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.192713    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-ef6be7\"/\"default-dockercfg-78mgg\""
Oct 30 19:03:22.194852 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.193077    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-ef6be7\"/\"kube-root-ca.crt\""
Oct 30 19:03:22.194852 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.193661    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-ef6be7\"/\"openshift-service-ca.crt\""
Oct 30 19:03:22.194852 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.193760    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-ef6be7\"/\"auth-test-model-ef6be7-kserve-self-signed-certs\""
Oct 30 19:03:22.194852 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.194017    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:22.206003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.205939    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"]
Oct 30 19:03:22.219415 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.219380    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-home\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.219558 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.219541    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-model-cache\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.219635 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.219608    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq5qk\" (UniqueName: \"kubernetes.io/projected/a3ce381c-876f-4c34-8403-196b4eef627c-kube-api-access-nq5qk\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.219770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.219743    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-tmp-dir\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.219903 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.219867    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-dshm\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.219973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.219931    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3ce381c-876f-4c34-8403-196b4eef627c-tls-certs\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.321457 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.321423    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-home\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.321641 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.321519    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-model-cache\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.321641 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.321573    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq5qk\" (UniqueName: \"kubernetes.io/projected/a3ce381c-876f-4c34-8403-196b4eef627c-kube-api-access-nq5qk\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.321783 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.321642    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-tmp-dir\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.321783 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.321743    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-dshm\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.321783 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.321767    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3ce381c-876f-4c34-8403-196b4eef627c-tls-certs\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.321953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.321931    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-home\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.322020 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.321957    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-model-cache\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.322020 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.322011    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-tmp-dir\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.324448 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.324381    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-dshm\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.324730 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.324710    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3ce381c-876f-4c34-8403-196b4eef627c-tls-certs\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.329690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.329664    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq5qk\" (UniqueName: \"kubernetes.io/projected/a3ce381c-876f-4c34-8403-196b4eef627c-kube-api-access-nq5qk\") pod \"auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") " pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.378067 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.378004    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:22.520733 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.520395    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:22.926427 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.926347    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:22.933670 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:22.933448    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"
Oct 30 19:03:23.064977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:23.064947    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"]
Oct 30 19:03:23.128584 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:23.128553    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz" event={"ID":"a3ce381c-876f-4c34-8403-196b4eef627c","Type":"ContainerStarted","Data":"2a388a1b7cac82a5ffe1dddda9e94c431ae2934ce9e1d5067d12b513d3b2cb64"}
Oct 30 19:03:23.664072 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:23.664041    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:24.006710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:24.006627    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:24.260492 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:24.260403    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz" event={"ID":"a3ce381c-876f-4c34-8403-196b4eef627c","Type":"ContainerStarted","Data":"eb89fae0c93e6b71e7d1cd5244ae8d41b1eacdb7a9bb9dbe1225dd8890601595"}
Oct 30 19:03:24.289141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:24.287626    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz" podStartSLOduration=2.287606086 podStartE2EDuration="2.287606086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:24.285525991 +0000 UTC m=+1626.001871069" watchObservedRunningTime="2026-10-30 19:03:24.287606086 +0000 UTC m=+1626.003951164"
Oct 30 19:03:24.522849 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:24.522766    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:24.535331 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:24.535292    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:03:24.535468 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:24.535363    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:03:24.728120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:24.727051    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:25.493613 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:25.493575    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:26.199153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:26.197150    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:26.284298 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:26.284257    4925 generic.go:356] "Generic (PLEG): container finished" podID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerID="5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8" exitCode=0
Oct 30 19:03:26.284298 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:26.284283    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" event={"ID":"f5760e51-b421-4e2b-8896-e61d63ef0e18","Type":"ContainerDied","Data":"5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8"}
Oct 30 19:03:26.285045 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:26.285027    4925 scope.go:122] "RemoveContainer" containerID="5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8"
Oct 30 19:03:26.286952 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:26.286919    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:03:26.287036 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:26.287012    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:03:26.287073 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:26.287064    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:03:26.941812 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:26.938652    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:27.295921 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:27.295840    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" event={"ID":"f5760e51-b421-4e2b-8896-e61d63ef0e18","Type":"ContainerStarted","Data":"49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9"}
Oct 30 19:03:27.436013 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:27.435979    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:03:27.436208 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:27.436181    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:03:27.436256 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:27.436233    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused"
Oct 30 19:03:27.963735 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:27.963695    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:28.436204 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:28.436163    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:03:28.436408 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:28.436224    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused"
Oct 30 19:03:28.807643 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:28.807542    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:29.436149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:29.436109    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:03:29.436565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:29.436172    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused"
Oct 30 19:03:29.578483 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:29.578446    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:30.245858 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:30.245827    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:30.295938 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:30.295906    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:30.364967 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:30.364737    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:03:30.439704 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:30.439664    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:03:30.440212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:30.439727    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused"
Oct 30 19:03:30.443279 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:30.443127    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:31.309120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:31.305483    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:31.435953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:31.435915    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:03:31.436150 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:31.436004    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused"
Oct 30 19:03:32.154117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:32.153694    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:32.436379 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:32.436342    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:03:32.436587 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:32.436408    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.167:15021/healthz/ready\": dial tcp 10.132.0.167:15021: connect: connection refused"
Oct 30 19:03:32.522051 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:32.522022    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:32.535672 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:32.535648    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:32.796177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:32.795778    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:33.409872 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:33.409843    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:03:33.449435 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:33.449403    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:03:33.450736 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:33.450715    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:03:33.452151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:33.452126    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:03:33.796234 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:33.793046    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:34.552195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:34.551965    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:35.269121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:35.268346    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:36.271123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:36.265117    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:36.702178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:36.701703    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:37.204521 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:37.203177    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:39.017132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:39.017055    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:39.192927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:39.192887    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:39.459233 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:39.459197    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:39.686558 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:39.686035    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:40.206865 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:40.206827    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:40.895119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:40.893750    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:41.560062 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:41.559937    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:42.362088 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:42.362056    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:43.293477 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:43.293425    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:43.306044 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:03:43.306012    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:03:44.463447 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:44.463413    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:45.167818 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:45.167779    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:45.369021 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:45.368982    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:45.978448 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:45.978397    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:46.602572 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:46.602538    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:47.398983 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:47.398946    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:47.991119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:47.983281    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:48.638121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:48.632750    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:49.253152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:49.253071    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:49.457119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:49.456963    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:50.170769 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.170724    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:50.600753 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.600683    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"]
Oct 30 19:03:50.605722 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.605697    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.627334 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.627307    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"]
Oct 30 19:03:50.701938 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.701909    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-e9ada0-openshift-default-dockercfg-dr8jc\""
Oct 30 19:03:50.738617 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.738590    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-podinfo\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.738800 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.738626    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-socket\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.738800 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.738756    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-token\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.738936 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.738817    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-certs\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.738936 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.738879    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-credential-socket\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.739051 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.738967    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-envoy\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.739112 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.739053    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-data\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.739252 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.739234    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9d5g\" (UniqueName: \"kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-kube-api-access-x9d5g\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.739303 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.739275    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd208996-61e2-49fe-b763-e55d63ae1d50-istiod-ca-cert\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.841242 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.841207    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9d5g\" (UniqueName: \"kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-kube-api-access-x9d5g\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.841443 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.841262    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd208996-61e2-49fe-b763-e55d63ae1d50-istiod-ca-cert\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.841443 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.841339    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-podinfo\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.841579 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.841460    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-socket\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.841641 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.841570    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-token\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.841641 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.841629    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-certs\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.841762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.841663    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-credential-socket\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.841822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.841793    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-envoy\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.841915 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.841898    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-data\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.842160 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.842078    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-certs\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.842311 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.842272    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd208996-61e2-49fe-b763-e55d63ae1d50-istiod-ca-cert\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.842389 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.842319    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-credential-socket\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.842565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.842539    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-data\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.842781 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.842757    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-socket\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.844239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.844219    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-podinfo\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.844726 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.844710    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-envoy\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.851238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.851179    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9d5g\" (UniqueName: \"kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-kube-api-access-x9d5g\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.851400 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.851384    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-token\") pod \"e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") " pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:50.922396 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:50.922367    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:51.155395 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:51.154068    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:51.230568 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:51.229637    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"]
Oct 30 19:03:51.232230 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:03:51.232189    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd208996_61e2_49fe_b763_e55d63ae1d50.slice/crio-4d6ae06828aa8a809ecccb3a00e45ec42a9582fbb50341850ff49d66b4ec87c9 WatchSource:0}: Error finding container 4d6ae06828aa8a809ecccb3a00e45ec42a9582fbb50341850ff49d66b4ec87c9: Status 404 returned error can't find the container with id 4d6ae06828aa8a809ecccb3a00e45ec42a9582fbb50341850ff49d66b4ec87c9
Oct 30 19:03:51.235668 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:51.235629    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:03:51.235757 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:51.235713    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:03:51.235803 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:51.235756    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:03:51.722276 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:51.722230    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q" event={"ID":"bd208996-61e2-49fe-b763-e55d63ae1d50","Type":"ContainerStarted","Data":"ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368"}
Oct 30 19:03:51.722463 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:51.722288    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q" event={"ID":"bd208996-61e2-49fe-b763-e55d63ae1d50","Type":"ContainerStarted","Data":"4d6ae06828aa8a809ecccb3a00e45ec42a9582fbb50341850ff49d66b4ec87c9"}
Oct 30 19:03:51.922647 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:51.922614    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:52.167122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:52.166198    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:52.924362 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:52.924325    4925 patch_prober.go:29] interesting pod/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.190:15021/healthz/ready\": dial tcp 10.132.0.190:15021: connect: connection refused" start-of-body=
Oct 30 19:03:52.924926 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:52.924388    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q" podUID="bd208996-61e2-49fe-b763-e55d63ae1d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.190:15021/healthz/ready\": dial tcp 10.132.0.190:15021: connect: connection refused"
Oct 30 19:03:53.086121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:53.085870    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:53.905259 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:53.905223    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:53.931787 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:53.931727    4925 patch_prober.go:29] interesting pod/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.190:15021/healthz/ready\": dial tcp 10.132.0.190:15021: connect: connection refused" start-of-body=
Oct 30 19:03:53.932432 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:53.932397    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q" podUID="bd208996-61e2-49fe-b763-e55d63ae1d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.190:15021/healthz/ready\": dial tcp 10.132.0.190:15021: connect: connection refused"
Oct 30 19:03:55.003063 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:55.002885    4925 patch_prober.go:29] interesting pod/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.190:15021/healthz/ready\": dial tcp 10.132.0.190:15021: connect: connection refused" start-of-body=
Oct 30 19:03:55.003753 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:55.003724    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q" podUID="bd208996-61e2-49fe-b763-e55d63ae1d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.190:15021/healthz/ready\": dial tcp 10.132.0.190:15021: connect: connection refused"
Oct 30 19:03:55.004583 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:55.004557    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:55.284653 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:55.284547    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:55.923698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:55.923664    4925 patch_prober.go:29] interesting pod/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.190:15021/healthz/ready\": dial tcp 10.132.0.190:15021: connect: connection refused" start-of-body=
Oct 30 19:03:55.923919 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:55.923717    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q" podUID="bd208996-61e2-49fe-b763-e55d63ae1d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.190:15021/healthz/ready\": dial tcp 10.132.0.190:15021: connect: connection refused"
Oct 30 19:03:56.193172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:56.192179    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:56.923621 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:56.923587    4925 patch_prober.go:29] interesting pod/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.190:15021/healthz/ready\": dial tcp 10.132.0.190:15021: connect: connection refused" start-of-body=
Oct 30 19:03:56.923793 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:56.923639    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q" podUID="bd208996-61e2-49fe-b763-e55d63ae1d50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.190:15021/healthz/ready\": dial tcp 10.132.0.190:15021: connect: connection refused"
Oct 30 19:03:57.089890 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:57.089854    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:57.905121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:57.902052    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:57.929275 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:57.929244    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:57.929726 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:57.929710    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:57.930392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:57.930372    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:03:57.959874 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:57.959825    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q" podStartSLOduration=7.959808526 podStartE2EDuration="7.959808526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:51.762052361 +0000 UTC m=+1653.478397438" watchObservedRunningTime="2026-10-30 19:03:57.959808526 +0000 UTC m=+1659.676153603"
Oct 30 19:03:58.570909 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:58.570812    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:59.054803 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:59.054770    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:59.630213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:03:59.629858    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:00.399120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:00.399016    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:01.053122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:01.052815    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:01.765838 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:01.765804    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:02.370688 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:02.370649    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:03.065206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:03.065169    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:03.787125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:03.785774    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:04.602223 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:04.599787    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:05.754781 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:05.754744    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:06.676050 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:06.674425    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:07.408087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:07.408049    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:08.256682 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:08.256631    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:08.960723 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:08.960686    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:09.368939 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:09.368855    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"]
Oct 30 19:04:09.528288 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:09.528245    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:09.702683 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:09.702589    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:09.984995 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:09.982145    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:10.261174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:10.261058    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:10.666467 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:10.666425    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:11.214927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:11.214894    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:11.419650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:11.419619    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:11.676777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:11.676714    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:11.916972 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:11.914965    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:12.122203 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:12.122089    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:12.818736 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:12.818612    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:13.304591 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:04:13.304552    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:04:13.414980 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:13.414887    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:14.181119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:14.180166    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:14.369713 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:14.369677    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh" podUID="6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4" containerName="payload-processing" containerID="cri-o://ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926" gracePeriod=25
Oct 30 19:04:14.728222 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:14.728191    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"
Oct 30 19:04:14.823249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:14.808313    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-plugins-config-volume\") pod \"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4\" (UID: \"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4\") "
Oct 30 19:04:14.823249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:14.816302    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-kube-api-access-2dz7g\" (UniqueName: \"kubernetes.io/projected/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-kube-api-access-2dz7g\") pod \"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4\" (UID: \"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4\") "
Oct 30 19:04:14.823249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:14.810921    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:14.823249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:14.810152    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-plugins-config-volume" pod "6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4" (UID: "6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:04:14.829343 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:14.828500    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-kube-api-access-2dz7g" pod "6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4" (UID: "6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4"). InnerVolumeSpecName "kube-api-access-2dz7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:14.918835 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:14.918800    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:14.918835 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:14.918831    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dz7g\" (UniqueName: \"kubernetes.io/projected/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4-kube-api-access-2dz7g\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:15.036003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:15.035969    4925 generic.go:356] "Generic (PLEG): container finished" podID="6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4" containerID="ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926" exitCode=0
Oct 30 19:04:15.036253 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:15.036038    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"
Oct 30 19:04:15.036253 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:15.036068    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh" event={"ID":"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4","Type":"ContainerDied","Data":"ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926"}
Oct 30 19:04:15.036253 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:15.036155    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh" event={"ID":"6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4","Type":"ContainerDied","Data":"a81616cc6b6d3811820a4b5c8d02208ef7ab3a6aca844d10c22191597994d87d"}
Oct 30 19:04:15.036253 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:15.036194    4925 scope.go:122] "RemoveContainer" containerID="ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926"
Oct 30 19:04:15.049870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:15.049849    4925 scope.go:122] "RemoveContainer" containerID="ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926"
Oct 30 19:04:15.050209 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:04:15.050179    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926\": container with ID starting with ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926 not found: ID does not exist" containerID="ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926"
Oct 30 19:04:15.050317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:15.050222    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926"} err="failed to get container status \"ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926\": rpc error: code = NotFound desc = could not find container \"ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926\": container with ID starting with ccc76cf3cd677e27191fde9f6088d15414dab8aa0a5bd3e4e644a932b9dc3926 not found: ID does not exist"
Oct 30 19:04:15.072016 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:15.071988    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"]
Oct 30 19:04:15.078239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:15.078185    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-e9ada0-689ff54b78-htzwh"]
Oct 30 19:04:15.481124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:15.477672    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:16.181192 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:16.181157    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:16.424777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:16.424747    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4" path="/var/lib/kubelet/pods/6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4/volumes"
Oct 30 19:04:16.858858 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:16.858824    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:17.497082 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:17.497045    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:18.254174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:18.254082    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:18.920821 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:18.920792    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:19.137559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:19.137518    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:19.141474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:19.141414    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"]
Oct 30 19:04:19.141745 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:19.141717    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q" podUID="bd208996-61e2-49fe-b763-e55d63ae1d50" containerName="istio-proxy" containerID="cri-o://ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368" gracePeriod=30
Oct 30 19:04:19.341171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:19.338864    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:19.910164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:19.910123    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:20.096036 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:20.095999    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:20.408190 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:20.407636    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:20.865171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:20.859211    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:21.045054 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.045020    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:21.599959 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.599925    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"]
Oct 30 19:04:21.601212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.601187    4925 cpu_manager.go:414] "containerMap: removing container" podUID="6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4" containerName="payload-processing"
Oct 30 19:04:21.601212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.601212    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4" containerName="payload-processing"
Oct 30 19:04:21.601482 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.601464    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f5f5b5b-3787-4149-84dd-fcc0a5f9b6a4" containerName="payload-processing"
Oct 30 19:04:21.607715 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.607692    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.612216 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.611806    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-2d1c71-openshift-default-dockercfg-vq6d2\""
Oct 30 19:04:21.614147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.614079    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"]
Oct 30 19:04:21.651249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.651225    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-envoy\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.651395 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.651276    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-token\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.651395 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.651300    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-socket\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.651485 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.651458    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-credential-socket\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.651522 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.651499    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-data\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.651609 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.651594    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzt46\" (UniqueName: \"kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-kube-api-access-tzt46\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.651679 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.651664    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-certs\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.651796 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.651771    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4d3263e-e6be-4c1e-9700-b654cedc6258-istiod-ca-cert\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.651885 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.651872    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-podinfo\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.665646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.665612    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:21.756124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.755245    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-credential-socket\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.756124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.755320    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-data\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.756124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.755416    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzt46\" (UniqueName: \"kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-kube-api-access-tzt46\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.756124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.755532    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-certs\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.756124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.755587    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4d3263e-e6be-4c1e-9700-b654cedc6258-istiod-ca-cert\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.756124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.755676    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-podinfo\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.756124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.755789    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-envoy\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.756124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.755845    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-token\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.756124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.755881    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-socket\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.756771 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.756464    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-socket\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.756771 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.756748    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-credential-socket\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.769118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.757043    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-data\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.769118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.757633    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-certs\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.769118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.758309    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4d3263e-e6be-4c1e-9700-b654cedc6258-istiod-ca-cert\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.769118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.766331    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-podinfo\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.774124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.769524    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-envoy\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.783806 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.775961    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-token\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.783806 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.777863    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzt46\" (UniqueName: \"kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-kube-api-access-tzt46\") pod \"e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") " pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:21.934327 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:21.934285    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:22.077155 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:22.077076    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:22.232484 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:04:22.232458    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d3263e_e6be_4c1e_9700_b654cedc6258.slice/crio-c4d51a2081dd4f243a0f4eaf80c8d2d438ff029f26fe7b6e9348103518b71652 WatchSource:0}: Error finding container c4d51a2081dd4f243a0f4eaf80c8d2d438ff029f26fe7b6e9348103518b71652: Status 404 returned error can't find the container with id c4d51a2081dd4f243a0f4eaf80c8d2d438ff029f26fe7b6e9348103518b71652
Oct 30 19:04:22.233269 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:22.233241    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"]
Oct 30 19:04:22.235630 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:22.235591    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:04:22.235704 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:22.235679    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:04:22.235762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:22.235717    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:04:22.257379 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:22.257349    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:22.452121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:22.447942    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:23.130176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:23.130070    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:23.219708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:23.219662    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2" event={"ID":"a4d3263e-e6be-4c1e-9700-b654cedc6258","Type":"ContainerStarted","Data":"f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1"}
Oct 30 19:04:23.219929 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:23.219742    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2" event={"ID":"a4d3263e-e6be-4c1e-9700-b654cedc6258","Type":"ContainerStarted","Data":"c4d51a2081dd4f243a0f4eaf80c8d2d438ff029f26fe7b6e9348103518b71652"}
Oct 30 19:04:23.240059 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:23.240005    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2" podStartSLOduration=2.239986832 podStartE2EDuration="2.239986832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:23.238671966 +0000 UTC m=+1684.955017043" watchObservedRunningTime="2026-10-30 19:04:23.239986832 +0000 UTC m=+1684.956331910"
Oct 30 19:04:23.821292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:23.821255    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:23.935109 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:23.935073    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:23.936477 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:23.936451    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.191:15021/healthz/ready\": dial tcp 10.132.0.191:15021: connect: connection refused" start-of-body=
Oct 30 19:04:23.936607 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:23.936516    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2" podUID="a4d3263e-e6be-4c1e-9700-b654cedc6258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.191:15021/healthz/ready\": dial tcp 10.132.0.191:15021: connect: connection refused"
Oct 30 19:04:24.128171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.128018    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:24.624044 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.623998    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:04:24.705452 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.705410    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd208996-61e2-49fe-b763-e55d63ae1d50-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd208996-61e2-49fe-b763-e55d63ae1d50-istiod-ca-cert\") pod \"bd208996-61e2-49fe-b763-e55d63ae1d50\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") "
Oct 30 19:04:24.705656 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.705491    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-socket\") pod \"bd208996-61e2-49fe-b763-e55d63ae1d50\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") "
Oct 30 19:04:24.705656 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.705543    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-certs\") pod \"bd208996-61e2-49fe-b763-e55d63ae1d50\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") "
Oct 30 19:04:24.705656 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.705627    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-podinfo\") pod \"bd208996-61e2-49fe-b763-e55d63ae1d50\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") "
Oct 30 19:04:24.706001 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.705676    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-data\") pod \"bd208996-61e2-49fe-b763-e55d63ae1d50\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") "
Oct 30 19:04:24.706001 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.705778    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-credential-socket\") pod \"bd208996-61e2-49fe-b763-e55d63ae1d50\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") "
Oct 30 19:04:24.706001 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.705869    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-kube-api-access-x9d5g\" (UniqueName: \"kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-kube-api-access-x9d5g\") pod \"bd208996-61e2-49fe-b763-e55d63ae1d50\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") "
Oct 30 19:04:24.706278 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.706008    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-token\" (UniqueName: \"kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-token\") pod \"bd208996-61e2-49fe-b763-e55d63ae1d50\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") "
Oct 30 19:04:24.706278 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.706069    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-envoy\") pod \"bd208996-61e2-49fe-b763-e55d63ae1d50\" (UID: \"bd208996-61e2-49fe-b763-e55d63ae1d50\") "
Oct 30 19:04:24.709441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.706473    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-credential-socket" pod "bd208996-61e2-49fe-b763-e55d63ae1d50" (UID: "bd208996-61e2-49fe-b763-e55d63ae1d50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:24.709441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.708165    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-socket" pod "bd208996-61e2-49fe-b763-e55d63ae1d50" (UID: "bd208996-61e2-49fe-b763-e55d63ae1d50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:24.709441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.708411    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-data" pod "bd208996-61e2-49fe-b763-e55d63ae1d50" (UID: "bd208996-61e2-49fe-b763-e55d63ae1d50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:24.709441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.708446    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd208996-61e2-49fe-b763-e55d63ae1d50-istiod-ca-cert" pod "bd208996-61e2-49fe-b763-e55d63ae1d50" (UID: "bd208996-61e2-49fe-b763-e55d63ae1d50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:04:24.709441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.708684    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-certs" pod "bd208996-61e2-49fe-b763-e55d63ae1d50" (UID: "bd208996-61e2-49fe-b763-e55d63ae1d50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:24.711850 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.711793    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.725043 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.723518    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-podinfo" pod "bd208996-61e2-49fe-b763-e55d63ae1d50" (UID: "bd208996-61e2-49fe-b763-e55d63ae1d50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:04:24.725043 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.724037    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-token" pod "bd208996-61e2-49fe-b763-e55d63ae1d50" (UID: "bd208996-61e2-49fe-b763-e55d63ae1d50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:24.726297 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.726077    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-envoy" pod "bd208996-61e2-49fe-b763-e55d63ae1d50" (UID: "bd208996-61e2-49fe-b763-e55d63ae1d50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:04:24.728383 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.728140    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-kube-api-access-x9d5g" pod "bd208996-61e2-49fe-b763-e55d63ae1d50" (UID: "bd208996-61e2-49fe-b763-e55d63ae1d50"). InnerVolumeSpecName "kube-api-access-x9d5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:04:24.818197 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.815085    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.818197 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.815170    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.818197 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.815198    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd208996-61e2-49fe-b763-e55d63ae1d50-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.820548 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.815213    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.820548 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.820474    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.820548 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.820494    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.820548 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.820509    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd208996-61e2-49fe-b763-e55d63ae1d50-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.820548 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.820523    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9d5g\" (UniqueName: \"kubernetes.io/projected/bd208996-61e2-49fe-b763-e55d63ae1d50-kube-api-access-x9d5g\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:04:24.847519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.847245    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:24.936131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.935682    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.191:15021/healthz/ready\": dial tcp 10.132.0.191:15021: connect: connection refused" start-of-body=
Oct 30 19:04:24.936131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:24.935766    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2" podUID="a4d3263e-e6be-4c1e-9700-b654cedc6258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.191:15021/healthz/ready\": dial tcp 10.132.0.191:15021: connect: connection refused"
Oct 30 19:04:25.283122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:25.282884    4925 generic.go:356] "Generic (PLEG): container finished" podID="bd208996-61e2-49fe-b763-e55d63ae1d50" containerID="ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368" exitCode=0
Oct 30 19:04:25.289225 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:25.284112    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"
Oct 30 19:04:25.289401 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:25.289314    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q" event={"ID":"bd208996-61e2-49fe-b763-e55d63ae1d50","Type":"ContainerDied","Data":"ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368"}
Oct 30 19:04:25.289484 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:25.289412    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q" event={"ID":"bd208996-61e2-49fe-b763-e55d63ae1d50","Type":"ContainerDied","Data":"4d6ae06828aa8a809ecccb3a00e45ec42a9582fbb50341850ff49d66b4ec87c9"}
Oct 30 19:04:25.289484 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:25.289446    4925 scope.go:122] "RemoveContainer" containerID="ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368"
Oct 30 19:04:25.307947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:25.307924    4925 scope.go:122] "RemoveContainer" containerID="ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368"
Oct 30 19:04:25.308249 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:04:25.308226    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368\": container with ID starting with ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368 not found: ID does not exist" containerID="ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368"
Oct 30 19:04:25.308338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:25.308258    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368"} err="failed to get container status \"ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368\": rpc error: code = NotFound desc = could not find container \"ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368\": container with ID starting with ad1079b5bcd15a9734a0a3bfc3916e50f56f995f5735122b0e08c6e27aeb2368 not found: ID does not exist"
Oct 30 19:04:25.319823 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:25.319797    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"]
Oct 30 19:04:25.323513 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:25.323493    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-e9ada0-openshift-default-685f6fdbfc-f2c9q"]
Oct 30 19:04:25.611258 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:25.608635    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:25.935065 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:25.935034    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.191:15021/healthz/ready\": dial tcp 10.132.0.191:15021: connect: connection refused" start-of-body=
Oct 30 19:04:25.935518 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:25.935088    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2" podUID="a4d3263e-e6be-4c1e-9700-b654cedc6258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.191:15021/healthz/ready\": dial tcp 10.132.0.191:15021: connect: connection refused"
Oct 30 19:04:26.395183 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:26.394520    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:26.468117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:26.461585    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd208996-61e2-49fe-b763-e55d63ae1d50" path="/var/lib/kubelet/pods/bd208996-61e2-49fe-b763-e55d63ae1d50/volumes"
Oct 30 19:04:26.934963 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:26.934928    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.191:15021/healthz/ready\": dial tcp 10.132.0.191:15021: connect: connection refused" start-of-body=
Oct 30 19:04:26.935163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:26.934978    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2" podUID="a4d3263e-e6be-4c1e-9700-b654cedc6258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.191:15021/healthz/ready\": dial tcp 10.132.0.191:15021: connect: connection refused"
Oct 30 19:04:27.091466 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:27.091432    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:27.880699 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:27.880650    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:27.936619 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:27.936568    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.191:15021/healthz/ready\": dial tcp 10.132.0.191:15021: connect: connection refused" start-of-body=
Oct 30 19:04:27.937516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:27.937347    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2" podUID="a4d3263e-e6be-4c1e-9700-b654cedc6258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.191:15021/healthz/ready\": dial tcp 10.132.0.191:15021: connect: connection refused"
Oct 30 19:04:28.709248 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:28.709214    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:29.026810 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:29.026724    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:29.027995 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:29.027970    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:29.029424 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:29.029399    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:04:29.489281 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:29.489244    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:30.281473 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:30.281438    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:31.070129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:31.069972    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:31.677270 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:31.677234    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:32.437871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:32.437832    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:33.123039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:33.123000    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:33.787019 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:33.786969    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:34.141116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:34.141002    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:34.462278 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:34.462245    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:34.659335 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:34.659291    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:34.873271 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:34.873180    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:35.065321 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:35.065278    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:35.620007 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:35.619974    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:36.425226 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:36.424984    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:37.326121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:37.320777    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:38.244137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:38.244082    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:39.119675 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:39.119631    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:39.811121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:39.810978    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:40.009465 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:40.009432    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:41.010117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:41.010065    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:41.841810 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:41.841763    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:42.675948 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:42.675913    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:43.306989 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:04:43.306948    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:04:43.415126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:43.413547    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:44.258231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:44.258195    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:45.027291 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:45.027263    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:45.819051 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:45.817500    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:46.625796 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:46.625631    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:47.416651 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:47.416618    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:48.201479 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:48.201440    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:49.156063 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:49.156026    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:49.878469 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:49.878408    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:50.113027 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:50.112984    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:51.004383 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:51.004339    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:51.840123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:51.838787    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:52.630120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:52.630024    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:53.563930 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:53.563899    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:54.420195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:54.419842    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:55.296870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:55.296829    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:56.125124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:56.119380    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:56.912692 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:56.912657    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:57.683880 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:57.683838    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:58.437046 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:58.437008    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:59.226171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:04:59.225373    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:00.016650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:00.016611    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:01.008156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:01.008117    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:01.756115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:01.756058    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:02.719126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:02.719060    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:03.420799 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:03.420764    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:04.276063 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:04.276027    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:05.133015 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:05.132912    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:05.883537 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:05.883499    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:06.688819 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:06.688778    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:07.405972 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:07.405932    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:08.371979 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:08.371917    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:09.226259 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:09.226225    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:10.228120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:10.225287    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:10.529181 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:10.529148    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:11.093132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:11.090259    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:11.512012 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:11.511966    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:11.779832 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:11.779733    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:12.505490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:12.505453    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:13.179743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:13.179709    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:13.303688 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:05:13.303647    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:05:13.969620 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:13.969589    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:14.690475 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:14.690440    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:15.390120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:15.389577    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:16.094119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:16.089422    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:17.015560 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:17.015526    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:18.164441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:18.164406    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:18.849886 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:18.849848    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:19.348325 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.348287    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7496-a16491"]
Oct 30 19:05:19.349067 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.349050    4925 cpu_manager.go:414] "containerMap: removing container" podUID="bd208996-61e2-49fe-b763-e55d63ae1d50" containerName="istio-proxy"
Oct 30 19:05:19.349163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.349067    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd208996-61e2-49fe-b763-e55d63ae1d50" containerName="istio-proxy"
Oct 30 19:05:19.349302 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.349289    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd208996-61e2-49fe-b763-e55d63ae1d50" containerName="istio-proxy"
Oct 30 19:05:19.353224 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.353207    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-a16491"
Oct 30 19:05:19.364199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.364164    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-c64f8\""
Oct 30 19:05:19.369981 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.369954    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7496-a16491"]
Oct 30 19:05:19.387983 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.387637    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4vx4\" (UniqueName: \"kubernetes.io/projected/31cf6240-757d-4ff8-81e0-974a963e6d23-kube-api-access-t4vx4\") pod \"test-curl-7496-a16491\" (UID: \"31cf6240-757d-4ff8-81e0-974a963e6d23\") " pod="opendatahub/test-curl-7496-a16491"
Oct 30 19:05:19.402549 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.402525    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:19.490558 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.490522    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4vx4\" (UniqueName: \"kubernetes.io/projected/31cf6240-757d-4ff8-81e0-974a963e6d23-kube-api-access-t4vx4\") pod \"test-curl-7496-a16491\" (UID: \"31cf6240-757d-4ff8-81e0-974a963e6d23\") " pod="opendatahub/test-curl-7496-a16491"
Oct 30 19:05:19.501202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.501170    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4vx4\" (UniqueName: \"kubernetes.io/projected/31cf6240-757d-4ff8-81e0-974a963e6d23-kube-api-access-t4vx4\") pod \"test-curl-7496-a16491\" (UID: \"31cf6240-757d-4ff8-81e0-974a963e6d23\") " pod="opendatahub/test-curl-7496-a16491"
Oct 30 19:05:19.681258 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.681219    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-a16491"
Oct 30 19:05:19.762880 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:19.762844    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:20.083020 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:05:20.082216    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31cf6240_757d_4ff8_81e0_974a963e6d23.slice/crio-111c7f9da5111e88d61d0691b0e256e308aa602875ffd00b416bc715732c299d WatchSource:0}: Error finding container 111c7f9da5111e88d61d0691b0e256e308aa602875ffd00b416bc715732c299d: Status 404 returned error can't find the container with id 111c7f9da5111e88d61d0691b0e256e308aa602875ffd00b416bc715732c299d
Oct 30 19:05:20.083020 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:20.082911    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7496-a16491"]
Oct 30 19:05:20.641868 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:20.641825    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:21.033257 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:21.033227    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7496-a16491" event={"ID":"31cf6240-757d-4ff8-81e0-974a963e6d23","Type":"ContainerStarted","Data":"2cfd5b67d91ac9d4507be6286a76b4408dfafa657fd4e1f4af8414f9ecb61f94"}
Oct 30 19:05:21.033446 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:21.033272    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7496-a16491" event={"ID":"31cf6240-757d-4ff8-81e0-974a963e6d23","Type":"ContainerStarted","Data":"111c7f9da5111e88d61d0691b0e256e308aa602875ffd00b416bc715732c299d"}
Oct 30 19:05:21.049377 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:21.049327    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7496-a16491" podStartSLOduration=2.049312696 podStartE2EDuration="2.049312696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:21.048419241 +0000 UTC m=+1742.764764315" watchObservedRunningTime="2026-10-30 19:05:21.049312696 +0000 UTC m=+1742.765657772"
Oct 30 19:05:21.328772 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:21.328650    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:21.855197 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:21.855130    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7496-a16491"]
Oct 30 19:05:21.860037 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:21.860008    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7496-a16491"]
Oct 30 19:05:22.014041 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.014008    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7496-2bba09"]
Oct 30 19:05:22.014837 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.014819    4925 cpu_manager.go:414] "containerMap: removing container" podUID="31cf6240-757d-4ff8-81e0-974a963e6d23" containerName="test-curl-7496-a16491"
Oct 30 19:05:22.014896 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.014838    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31cf6240-757d-4ff8-81e0-974a963e6d23" containerName="test-curl-7496-a16491"
Oct 30 19:05:22.014981 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.014972    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="31cf6240-757d-4ff8-81e0-974a963e6d23" containerName="test-curl-7496-a16491"
Oct 30 19:05:22.029164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.019215    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-2bba09"
Oct 30 19:05:22.114203 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.097013    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:22.128295 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.128244    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7496-2bba09"]
Oct 30 19:05:22.164303 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.164252    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79v8t\" (UniqueName: \"kubernetes.io/projected/e112a623-bc66-4700-9796-71634451f285-kube-api-access-79v8t\") pod \"test-curl-7496-2bba09\" (UID: \"e112a623-bc66-4700-9796-71634451f285\") " pod="opendatahub/test-curl-7496-2bba09"
Oct 30 19:05:22.265627 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.265597    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79v8t\" (UniqueName: \"kubernetes.io/projected/e112a623-bc66-4700-9796-71634451f285-kube-api-access-79v8t\") pod \"test-curl-7496-2bba09\" (UID: \"e112a623-bc66-4700-9796-71634451f285\") " pod="opendatahub/test-curl-7496-2bba09"
Oct 30 19:05:22.274856 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.274826    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79v8t\" (UniqueName: \"kubernetes.io/projected/e112a623-bc66-4700-9796-71634451f285-kube-api-access-79v8t\") pod \"test-curl-7496-2bba09\" (UID: \"e112a623-bc66-4700-9796-71634451f285\") " pod="opendatahub/test-curl-7496-2bba09"
Oct 30 19:05:22.442010 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.441979    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-2bba09"
Oct 30 19:05:22.784013 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.783978    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:22.802972 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:22.802935    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7496-2bba09"]
Oct 30 19:05:23.093808 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:23.093716    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7496-2bba09" event={"ID":"e112a623-bc66-4700-9796-71634451f285","Type":"ContainerStarted","Data":"a4dccfabd115ae32583385f831cbb921b920224ba9cf8c60c2744b3f231efc0e"}
Oct 30 19:05:23.093808 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:23.093764    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7496-2bba09" event={"ID":"e112a623-bc66-4700-9796-71634451f285","Type":"ContainerStarted","Data":"c5b1fe086b7f66c820e84f3ae8ba3d2ce207f616236bd353424ca04a9af6b985"}
Oct 30 19:05:23.094404 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:23.093779    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7496-a16491" podUID="31cf6240-757d-4ff8-81e0-974a963e6d23" containerName="test-curl-7496-a16491" containerID="cri-o://2cfd5b67d91ac9d4507be6286a76b4408dfafa657fd4e1f4af8414f9ecb61f94" gracePeriod=2
Oct 30 19:05:23.110763 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:23.110718    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7496-2bba09" podStartSLOduration=2.110706335 podStartE2EDuration="2.110706335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:23.107533485 +0000 UTC m=+1744.823878555" watchObservedRunningTime="2026-10-30 19:05:23.110706335 +0000 UTC m=+1744.827051411"
Oct 30 19:05:23.518933 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:23.515737    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:23.795679 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:23.794893    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7496-2bba09"]
Oct 30 19:05:23.801201 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:23.801169    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7496-2bba09"]
Oct 30 19:05:24.191129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:24.185401    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:24.462266 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:24.462179    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:24.871445 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:24.871341    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:25.083118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:25.082606    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:25.220598 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:25.220426    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7496-2bba09" podUID="e112a623-bc66-4700-9796-71634451f285" containerName="test-curl-7496-2bba09" containerID="cri-o://a4dccfabd115ae32583385f831cbb921b920224ba9cf8c60c2744b3f231efc0e" gracePeriod=2
Oct 30 19:05:25.449834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:25.449811    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-a16491"
Oct 30 19:05:25.575437 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:25.575404    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31cf6240-757d-4ff8-81e0-974a963e6d23-kube-api-access-t4vx4\" (UniqueName: \"kubernetes.io/projected/31cf6240-757d-4ff8-81e0-974a963e6d23-kube-api-access-t4vx4\") pod \"31cf6240-757d-4ff8-81e0-974a963e6d23\" (UID: \"31cf6240-757d-4ff8-81e0-974a963e6d23\") "
Oct 30 19:05:25.577802 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:25.577777    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31cf6240-757d-4ff8-81e0-974a963e6d23-kube-api-access-t4vx4" pod "31cf6240-757d-4ff8-81e0-974a963e6d23" (UID: "31cf6240-757d-4ff8-81e0-974a963e6d23"). InnerVolumeSpecName "kube-api-access-t4vx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:25.677592 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:25.677504    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4vx4\" (UniqueName: \"kubernetes.io/projected/31cf6240-757d-4ff8-81e0-974a963e6d23-kube-api-access-t4vx4\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:25.802122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:25.801725    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:26.284119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:26.279051    4925 generic.go:356] "Generic (PLEG): container finished" podID="31cf6240-757d-4ff8-81e0-974a963e6d23" containerID="2cfd5b67d91ac9d4507be6286a76b4408dfafa657fd4e1f4af8414f9ecb61f94" exitCode=137
Oct 30 19:05:26.284119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:26.279167    4925 scope.go:122] "RemoveContainer" containerID="2cfd5b67d91ac9d4507be6286a76b4408dfafa657fd4e1f4af8414f9ecb61f94"
Oct 30 19:05:26.284119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:26.279360    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-a16491"
Oct 30 19:05:26.317907 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:26.317884    4925 scope.go:122] "RemoveContainer" containerID="2cfd5b67d91ac9d4507be6286a76b4408dfafa657fd4e1f4af8414f9ecb61f94"
Oct 30 19:05:26.318884 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:05:26.318811    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cfd5b67d91ac9d4507be6286a76b4408dfafa657fd4e1f4af8414f9ecb61f94\": container with ID starting with 2cfd5b67d91ac9d4507be6286a76b4408dfafa657fd4e1f4af8414f9ecb61f94 not found: ID does not exist" containerID="2cfd5b67d91ac9d4507be6286a76b4408dfafa657fd4e1f4af8414f9ecb61f94"
Oct 30 19:05:26.318884 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:26.318849    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cfd5b67d91ac9d4507be6286a76b4408dfafa657fd4e1f4af8414f9ecb61f94"} err="failed to get container status \"2cfd5b67d91ac9d4507be6286a76b4408dfafa657fd4e1f4af8414f9ecb61f94\": rpc error: code = NotFound desc = could not find container \"2cfd5b67d91ac9d4507be6286a76b4408dfafa657fd4e1f4af8414f9ecb61f94\": container with ID starting with 2cfd5b67d91ac9d4507be6286a76b4408dfafa657fd4e1f4af8414f9ecb61f94 not found: ID does not exist"
Oct 30 19:05:26.319862 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:05:26.319830    4925 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7496-a16491\" is forbidden: User \"system:node:ip-10-0-134-58.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-58.us-east-2.compute.internal' and this object" podUID="31cf6240-757d-4ff8-81e0-974a963e6d23" pod="opendatahub/test-curl-7496-a16491"
Oct 30 19:05:26.424290 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:26.424261    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31cf6240-757d-4ff8-81e0-974a963e6d23" path="/var/lib/kubelet/pods/31cf6240-757d-4ff8-81e0-974a963e6d23/volumes"
Oct 30 19:05:26.639504 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:26.639409    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:27.332814 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:27.327633    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:27.585817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:27.585784    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-2bba09"
Oct 30 19:05:27.738004 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:27.737959    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e112a623-bc66-4700-9796-71634451f285-kube-api-access-79v8t\" (UniqueName: \"kubernetes.io/projected/e112a623-bc66-4700-9796-71634451f285-kube-api-access-79v8t\") pod \"e112a623-bc66-4700-9796-71634451f285\" (UID: \"e112a623-bc66-4700-9796-71634451f285\") "
Oct 30 19:05:27.740255 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:27.740232    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e112a623-bc66-4700-9796-71634451f285-kube-api-access-79v8t" pod "e112a623-bc66-4700-9796-71634451f285" (UID: "e112a623-bc66-4700-9796-71634451f285"). InnerVolumeSpecName "kube-api-access-79v8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:27.840608 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:27.840528    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79v8t\" (UniqueName: \"kubernetes.io/projected/e112a623-bc66-4700-9796-71634451f285-kube-api-access-79v8t\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:05:28.263561 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:28.263517    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:28.340718 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:28.340680    4925 generic.go:356] "Generic (PLEG): container finished" podID="e112a623-bc66-4700-9796-71634451f285" containerID="a4dccfabd115ae32583385f831cbb921b920224ba9cf8c60c2744b3f231efc0e" exitCode=137
Oct 30 19:05:28.341189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:28.340760    4925 scope.go:122] "RemoveContainer" containerID="a4dccfabd115ae32583385f831cbb921b920224ba9cf8c60c2744b3f231efc0e"
Oct 30 19:05:28.341189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:28.340809    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7496-2bba09"
Oct 30 19:05:28.352606 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:28.352434    4925 scope.go:122] "RemoveContainer" containerID="a4dccfabd115ae32583385f831cbb921b920224ba9cf8c60c2744b3f231efc0e"
Oct 30 19:05:28.352782 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:05:28.352762    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4dccfabd115ae32583385f831cbb921b920224ba9cf8c60c2744b3f231efc0e\": container with ID starting with a4dccfabd115ae32583385f831cbb921b920224ba9cf8c60c2744b3f231efc0e not found: ID does not exist" containerID="a4dccfabd115ae32583385f831cbb921b920224ba9cf8c60c2744b3f231efc0e"
Oct 30 19:05:28.352840 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:28.352789    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4dccfabd115ae32583385f831cbb921b920224ba9cf8c60c2744b3f231efc0e"} err="failed to get container status \"a4dccfabd115ae32583385f831cbb921b920224ba9cf8c60c2744b3f231efc0e\": rpc error: code = NotFound desc = could not find container \"a4dccfabd115ae32583385f831cbb921b920224ba9cf8c60c2744b3f231efc0e\": container with ID starting with a4dccfabd115ae32583385f831cbb921b920224ba9cf8c60c2744b3f231efc0e not found: ID does not exist"
Oct 30 19:05:28.353268 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:05:28.353244    4925 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7496-2bba09\" is forbidden: User \"system:node:ip-10-0-134-58.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-58.us-east-2.compute.internal' and this object" podUID="e112a623-bc66-4700-9796-71634451f285" pod="opendatahub/test-curl-7496-2bba09"
Oct 30 19:05:28.422410 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:05:28.422375    4925 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7496-2bba09\" is forbidden: User \"system:node:ip-10-0-134-58.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-58.us-east-2.compute.internal' and this object" podUID="e112a623-bc66-4700-9796-71634451f285" pod="opendatahub/test-curl-7496-2bba09"
Oct 30 19:05:28.424717 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:28.424686    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e112a623-bc66-4700-9796-71634451f285" path="/var/lib/kubelet/pods/e112a623-bc66-4700-9796-71634451f285/volumes"
Oct 30 19:05:28.519550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:28.519456    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:28.893004 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:28.892890    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:29.628737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:29.628687    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:29.860188 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:29.860005    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:30.169830 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:30.169729    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:31.224352 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:31.224300    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:31.482210 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:31.480573    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:31.814177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:31.814005    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:32.505131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:32.502489    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:33.239443 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:33.239410    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:34.044184 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:34.044147    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:34.706916 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:34.706882    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:35.366173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:35.365716    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:35.995432 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:35.995340    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:36.702650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:36.702616    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:37.434117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:37.428547    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:38.107526 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:38.106686    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:38.975126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:38.971209    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:39.158543 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:39.158507    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:39.338889 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:39.338806    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:40.037773 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:40.037728    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:40.696148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:40.695705    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:41.368202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:41.368170    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:41.987612 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:41.987576    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:42.734122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:42.728283    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:43.331123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:43.330924    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:43.385573 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:05:43.385521    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:05:43.998784 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:43.998747    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:44.788134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:44.777432    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:45.382119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:45.378351    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:46.084128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:46.078679    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:46.681120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:46.679736    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:47.355779 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:47.355741    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:47.961509 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:47.961474    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:48.797790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:48.797755    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:49.573640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:49.573312    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:49.991653 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:49.991614    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:50.185289 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:50.185258    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:50.546599 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:50.546565    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:51.207591 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:51.207558    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:51.906258 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:51.906170    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:52.547688 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:52.547653    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:53.202776 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:53.202741    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:53.700121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:53.699947    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:54.067808 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:54.067721    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:54.241321 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:54.241285    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:54.558654 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:54.558619    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:54.996138 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:54.994490    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:55.243076 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:55.243027    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:55.906465 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:55.905690    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:56.604573 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:56.604536    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:57.387124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:57.382543    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:57.911777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:57.911744    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:58.108408 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:58.108354    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:58.558997 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:58.558961    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:59.506813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:59.506775    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:59.714060 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:05:59.714011    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:00.708316 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:00.708286    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:01.298125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:01.294674    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:01.916298 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:01.916265    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:02.684958 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:02.684921    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:03.395996 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:03.379461    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:04.059254 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:04.059218    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:04.680741 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:04.680703    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:05.287119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:05.283864    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:05.976625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:05.976589    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:06.602040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:06.602011    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:07.293755 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:07.293722    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:08.036265 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.036229    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:08.736414 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.736378    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"]
Oct 30 19:06:08.737880 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.737853    4925 cpu_manager.go:414] "containerMap: removing container" podUID="e112a623-bc66-4700-9796-71634451f285" containerName="test-curl-7496-2bba09"
Oct 30 19:06:08.737880 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.737880    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e112a623-bc66-4700-9796-71634451f285" containerName="test-curl-7496-2bba09"
Oct 30 19:06:08.738073 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.738061    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="e112a623-bc66-4700-9796-71634451f285" containerName="test-curl-7496-2bba09"
Oct 30 19:06:08.742244 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.742178    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:08.752111 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.748393    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-46385c-serving-cert\""
Oct 30 19:06:08.755331 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.755308    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"]
Oct 30 19:06:08.769323 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.769291    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5906e228-28c9-4e6d-9a10-c4336f9d133d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:08.769475 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.769450    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:08.769589 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.769565    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvd7h\" (UniqueName: \"kubernetes.io/projected/5906e228-28c9-4e6d-9a10-c4336f9d133d-kube-api-access-lvd7h\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:08.769740 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.769718    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:08.901119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.896230    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:08.901119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.896304    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvd7h\" (UniqueName: \"kubernetes.io/projected/5906e228-28c9-4e6d-9a10-c4336f9d133d-kube-api-access-lvd7h\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:08.901119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.896447    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:08.901119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.896611    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5906e228-28c9-4e6d-9a10-c4336f9d133d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:08.930133 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:06:08.929630    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-serving-cert: secret "maas-api-e2e-shared-b-w1-46385c-serving-cert" not found
Oct 30 19:06:08.930133 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:06:08.929709    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-maas-api-tls podName:5906e228-28c9-4e6d-9a10-c4336f9d133d nodeName:}" failed. No retries permitted until 2026-10-30 19:06:09.429687797 +0000 UTC m=+1791.146032865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-maas-api-tls") pod "maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c" (UID: "5906e228-28c9-4e6d-9a10-c4336f9d133d") : secret "maas-api-e2e-shared-b-w1-46385c-serving-cert" not found
Oct 30 19:06:08.935030 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.932664    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5906e228-28c9-4e6d-9a10-c4336f9d133d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:08.935030 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.934826    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:08.985412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.984178    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvd7h\" (UniqueName: \"kubernetes.io/projected/5906e228-28c9-4e6d-9a10-c4336f9d133d-kube-api-access-lvd7h\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:08.985412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:08.984848    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:09.109917 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.109837    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:09.323463 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.323428    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:09.516679 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.508926    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:09.540127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.539122    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:09.683516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.683480    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:09.768464 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.768370    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:09.891157 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.891084    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"]
Oct 30 19:06:09.898284 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.898254    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:09.908188 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.905227    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-dcd326-openshift-default-dockercfg-hmw94\""
Oct 30 19:06:09.912086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.912062    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"]
Oct 30 19:06:09.940467 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.940430    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-data\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:09.940615 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.940512    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-credential-socket\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:09.940615 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.940545    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-certs\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:09.940615 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.940603    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-socket\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:09.940816 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.940646    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mthdq\" (UniqueName: \"kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-kube-api-access-mthdq\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:09.940816 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.940693    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e449af3e-0d2b-4084-a82e-34752431dae9-istiod-ca-cert\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:09.940816 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.940723    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e449af3e-0d2b-4084-a82e-34752431dae9-istio-podinfo\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:09.940978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.940830    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-envoy\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:09.940978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:09.940914    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-istio-token\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.042665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.042587    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-socket\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.042665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.042649    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mthdq\" (UniqueName: \"kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-kube-api-access-mthdq\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.042871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.042686    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e449af3e-0d2b-4084-a82e-34752431dae9-istiod-ca-cert\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.042871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.042712    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e449af3e-0d2b-4084-a82e-34752431dae9-istio-podinfo\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.042871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.042827    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-envoy\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.042981 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.042946    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-istio-token\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.043061 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.043031    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-socket\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.043202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.043063    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-data\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.043202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.043159    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-credential-socket\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.043202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.043191    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-certs\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.043442 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.043421    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-data\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.043506 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.043491    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-credential-socket\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.043550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.043507    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e449af3e-0d2b-4084-a82e-34752431dae9-istiod-ca-cert\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.043596 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.043579    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-certs\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.045431 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.045408    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-envoy\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.045661 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.045645    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e449af3e-0d2b-4084-a82e-34752431dae9-istio-podinfo\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.051392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.051367    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-istio-token\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.051559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.051511    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mthdq\" (UniqueName: \"kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-kube-api-access-mthdq\") pod \"e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") " pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.085692 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.085665    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"]
Oct 30 19:06:10.086781 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:06:10.086758    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5906e228_28c9_4e6d_9a10_c4336f9d133d.slice/crio-984294bd9f1d95937d10edadd7aaf78a059ab09f64fbfca82b131e03b46a64bc WatchSource:0}: Error finding container 984294bd9f1d95937d10edadd7aaf78a059ab09f64fbfca82b131e03b46a64bc: Status 404 returned error can't find the container with id 984294bd9f1d95937d10edadd7aaf78a059ab09f64fbfca82b131e03b46a64bc
Oct 30 19:06:10.087660 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.087641    4925 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:06:10.228317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.228284    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:10.526749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.526712    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"]
Oct 30 19:06:10.527028 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:06:10.527006    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode449af3e_0d2b_4084_a82e_34752431dae9.slice/crio-78f80493e8096d872dc635d1352cc9f48d80b7594f675355f0e16ede4267e88b WatchSource:0}: Error finding container 78f80493e8096d872dc635d1352cc9f48d80b7594f675355f0e16ede4267e88b: Status 404 returned error can't find the container with id 78f80493e8096d872dc635d1352cc9f48d80b7594f675355f0e16ede4267e88b
Oct 30 19:06:10.535620 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.535584    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:06:10.535729 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.535666    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:06:10.535729 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.535697    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:06:10.562924 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.562899    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:10.747117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:10.745284    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:11.090039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:11.090001    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp" event={"ID":"e449af3e-0d2b-4084-a82e-34752431dae9","Type":"ContainerStarted","Data":"4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a"}
Oct 30 19:06:11.090674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:11.090649    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp" event={"ID":"e449af3e-0d2b-4084-a82e-34752431dae9","Type":"ContainerStarted","Data":"78f80493e8096d872dc635d1352cc9f48d80b7594f675355f0e16ede4267e88b"}
Oct 30 19:06:11.092217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:11.092162    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c" event={"ID":"5906e228-28c9-4e6d-9a10-c4336f9d133d","Type":"ContainerStarted","Data":"2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5"}
Oct 30 19:06:11.092217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:11.092217    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c" event={"ID":"5906e228-28c9-4e6d-9a10-c4336f9d133d","Type":"ContainerStarted","Data":"984294bd9f1d95937d10edadd7aaf78a059ab09f64fbfca82b131e03b46a64bc"}
Oct 30 19:06:11.092516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:11.092490    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:11.122713 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:11.122667    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp" podStartSLOduration=2.122655333 podStartE2EDuration="2.122655333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:11.12093935 +0000 UTC m=+1792.837284426" watchObservedRunningTime="2026-10-30 19:06:11.122655333 +0000 UTC m=+1792.839000483"
Oct 30 19:06:11.153213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:11.153163    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c" podStartSLOduration=2.717446253 podStartE2EDuration="3.153144969s" totalImagesPullingTime="435.698716ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:11.150742698 +0000 UTC m=+1792.867087775" watchObservedRunningTime="2026-10-30 19:06:11.153144969 +0000 UTC m=+1792.869490047"
Oct 30 19:06:11.228607 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:11.228572    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:11.230479 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:11.230452    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.195:15021/healthz/ready\": dial tcp 10.132.0.195:15021: connect: connection refused" start-of-body=
Oct 30 19:06:11.230590 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:11.230503    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp" podUID="e449af3e-0d2b-4084-a82e-34752431dae9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.195:15021/healthz/ready\": dial tcp 10.132.0.195:15021: connect: connection refused"
Oct 30 19:06:11.801124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:11.800649    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:12.229136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:12.229086    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.195:15021/healthz/ready\": dial tcp 10.132.0.195:15021: connect: connection refused" start-of-body=
Oct 30 19:06:12.229631 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:12.229158    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp" podUID="e449af3e-0d2b-4084-a82e-34752431dae9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.195:15021/healthz/ready\": dial tcp 10.132.0.195:15021: connect: connection refused"
Oct 30 19:06:12.720116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:12.709531    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:13.229452 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:13.229408    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.195:15021/healthz/ready\": dial tcp 10.132.0.195:15021: connect: connection refused" start-of-body=
Oct 30 19:06:13.229893 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:13.229474    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp" podUID="e449af3e-0d2b-4084-a82e-34752431dae9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.195:15021/healthz/ready\": dial tcp 10.132.0.195:15021: connect: connection refused"
Oct 30 19:06:13.323500 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:06:13.323463    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:06:13.520179 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:13.519798    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:14.229546 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:14.229511    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.195:15021/healthz/ready\": dial tcp 10.132.0.195:15021: connect: connection refused" start-of-body=
Oct 30 19:06:14.230147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:14.229571    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp" podUID="e449af3e-0d2b-4084-a82e-34752431dae9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.195:15021/healthz/ready\": dial tcp 10.132.0.195:15021: connect: connection refused"
Oct 30 19:06:14.347126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:14.346206    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:15.248127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:15.247556    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.195:15021/healthz/ready\": dial tcp 10.132.0.195:15021: connect: connection refused" start-of-body=
Oct 30 19:06:15.248127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:15.247638    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp" podUID="e449af3e-0d2b-4084-a82e-34752431dae9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.195:15021/healthz/ready\": dial tcp 10.132.0.195:15021: connect: connection refused"
Oct 30 19:06:15.316121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:15.312768    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:16.090121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:16.089921    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:16.229500 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:16.229471    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.195:15021/healthz/ready\": dial tcp 10.132.0.195:15021: connect: connection refused" start-of-body=
Oct 30 19:06:16.229750 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:16.229728    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp" podUID="e449af3e-0d2b-4084-a82e-34752431dae9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.195:15021/healthz/ready\": dial tcp 10.132.0.195:15021: connect: connection refused"
Oct 30 19:06:16.909446 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:16.909398    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:17.106532 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:17.106495    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:06:17.233825 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:17.233752    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:17.234231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:17.234202    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:17.234942 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:17.234921    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:06:17.877475 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:17.877447    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:18.977153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:18.976630    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:19.771072 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:19.771036    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:20.503928 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:20.503888    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:21.243182 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:21.243141    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:21.961689 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:21.961651    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:22.187868 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:22.187837    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 19:06:22.253813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:22.253732    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 19:06:22.885073 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:22.885030    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:23.821003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:23.820969    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:24.646710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:24.646675    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:25.289265 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:25.289228    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:25.941496 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:25.927113    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:26.699141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:26.693654    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:27.306389 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:27.306352    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:28.014435 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:28.014401    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:28.194119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:28.191320    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:28.796718 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:28.796682    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:29.369534 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:29.369491    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:29.966668 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:29.966631    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:30.450909 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:30.450858    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:30.705776 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:30.705692    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:30.959176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:30.950819    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:31.466368 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:31.466311    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:31.899638 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:31.895788    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:32.760290 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:32.760256    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:33.586541 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:33.586506    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:34.270733 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:34.270697    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:34.559717 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:34.559597    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:34.831317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:34.831236    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:35.193321 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.191019    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:35.639647 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.639578    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"]
Oct 30 19:06:35.655292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.655260    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"]
Oct 30 19:06:35.655614 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.655596    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"
Oct 30 19:06:35.667049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.667022    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-ca2cd5-dockercfg-7r8z2\""
Oct 30 19:06:35.667342 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.667326    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-ca2cd5\""
Oct 30 19:06:35.686983 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.686951    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:35.710203 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.710171    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"]
Oct 30 19:06:35.716394 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.716373    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"
Oct 30 19:06:35.727804 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.727774    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"]
Oct 30 19:06:35.923177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.911403    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk\" (UID: \"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"
Oct 30 19:06:35.923177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.911482    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/829ee262-f7f8-495c-ba74-1a6e68f79b2f-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j\" (UID: \"829ee262-f7f8-495c-ba74-1a6e68f79b2f\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"
Oct 30 19:06:35.923177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.911616    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkcxz\" (UniqueName: \"kubernetes.io/projected/829ee262-f7f8-495c-ba74-1a6e68f79b2f-kube-api-access-xkcxz\") pod \"payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j\" (UID: \"829ee262-f7f8-495c-ba74-1a6e68f79b2f\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"
Oct 30 19:06:35.923177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:35.911885    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcn62\" (UniqueName: \"kubernetes.io/projected/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-kube-api-access-hcn62\") pod \"payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk\" (UID: \"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"
Oct 30 19:06:36.032539 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.015798    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:36.032539 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.016972    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcn62\" (UniqueName: \"kubernetes.io/projected/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-kube-api-access-hcn62\") pod \"payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk\" (UID: \"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"
Oct 30 19:06:36.032539 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.017069    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk\" (UID: \"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"
Oct 30 19:06:36.032539 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.017148    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/829ee262-f7f8-495c-ba74-1a6e68f79b2f-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j\" (UID: \"829ee262-f7f8-495c-ba74-1a6e68f79b2f\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"
Oct 30 19:06:36.032539 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.017270    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkcxz\" (UniqueName: \"kubernetes.io/projected/829ee262-f7f8-495c-ba74-1a6e68f79b2f-kube-api-access-xkcxz\") pod \"payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j\" (UID: \"829ee262-f7f8-495c-ba74-1a6e68f79b2f\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"
Oct 30 19:06:36.032539 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.018832    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk\" (UID: \"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"
Oct 30 19:06:36.032539 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.024991    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/829ee262-f7f8-495c-ba74-1a6e68f79b2f-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j\" (UID: \"829ee262-f7f8-495c-ba74-1a6e68f79b2f\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"
Oct 30 19:06:36.099270 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.096216    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkcxz\" (UniqueName: \"kubernetes.io/projected/829ee262-f7f8-495c-ba74-1a6e68f79b2f-kube-api-access-xkcxz\") pod \"payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j\" (UID: \"829ee262-f7f8-495c-ba74-1a6e68f79b2f\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"
Oct 30 19:06:36.102404 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.101817    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcn62\" (UniqueName: \"kubernetes.io/projected/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-kube-api-access-hcn62\") pod \"payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk\" (UID: \"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"
Oct 30 19:06:36.302108 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.301777    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"
Oct 30 19:06:36.311841 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.311795    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:36.348565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.348526    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"
Oct 30 19:06:36.683315 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.683286    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"]
Oct 30 19:06:36.716557 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:06:36.716519    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod829ee262_f7f8_495c_ba74_1a6e68f79b2f.slice/crio-540789c29b379bea3415def9e451356e4aa1df2f92d9842b5c1940206d5c142f WatchSource:0}: Error finding container 540789c29b379bea3415def9e451356e4aa1df2f92d9842b5c1940206d5c142f: Status 404 returned error can't find the container with id 540789c29b379bea3415def9e451356e4aa1df2f92d9842b5c1940206d5c142f
Oct 30 19:06:36.717595 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.717569    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"]
Oct 30 19:06:36.837299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:36.837260    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:37.048172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:37.046303    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:37.529047 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:37.529012    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j" event={"ID":"829ee262-f7f8-495c-ba74-1a6e68f79b2f","Type":"ContainerStarted","Data":"463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2"}
Oct 30 19:06:37.529270 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:37.529086    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j" event={"ID":"829ee262-f7f8-495c-ba74-1a6e68f79b2f","Type":"ContainerStarted","Data":"540789c29b379bea3415def9e451356e4aa1df2f92d9842b5c1940206d5c142f"}
Oct 30 19:06:37.529270 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:37.529141    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"
Oct 30 19:06:37.530908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:37.530881    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk" event={"ID":"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3","Type":"ContainerStarted","Data":"3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9"}
Oct 30 19:06:37.531050 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:37.530926    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk" event={"ID":"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3","Type":"ContainerStarted","Data":"83c0b3634a1523f474d74b5e90dc08f5ecd77cfda36186863dccea9a91336aed"}
Oct 30 19:06:37.531050 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:37.530952    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"
Oct 30 19:06:37.548249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:37.548206    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j" podStartSLOduration=2.5481932499999997 podStartE2EDuration="2.54819325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:37.546232095 +0000 UTC m=+1819.262577170" watchObservedRunningTime="2026-10-30 19:06:37.54819325 +0000 UTC m=+1819.264538340"
Oct 30 19:06:37.567237 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:37.567186    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk" podStartSLOduration=2.567169316 podStartE2EDuration="2.567169316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:37.566493635 +0000 UTC m=+1819.282838729" watchObservedRunningTime="2026-10-30 19:06:37.567169316 +0000 UTC m=+1819.283514394"
Oct 30 19:06:37.725125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:37.716030    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:38.484590 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:38.482406    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:39.197690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:39.196519    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:40.004132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:39.999304    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:40.678717 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:40.678664    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:41.487119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:41.482421    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:42.269149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:42.265687    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:42.889291 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:42.889256    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:43.449841 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:43.449746    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:43.452561 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:06:43.452528    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:06:43.844466 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:43.844373    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"]
Oct 30 19:06:43.845668 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:43.845640    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:44.029960 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:44.029928    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:44.634072 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:44.633712    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:45.386397 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:45.386344    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:45.573623 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:45.573583    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:45.921140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:45.914829    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:46.226251 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.216998    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"]
Oct 30 19:06:46.237536 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.237507    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.247852 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.246545    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-ed012e-openshift-default-dockercfg-ls79r\""
Oct 30 19:06:46.247852 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.246547    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"]
Oct 30 19:06:46.370348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.367999    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:46.378706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.371839    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-token\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.378706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.371920    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-data\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.378706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.371968    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pggxk\" (UniqueName: \"kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-kube-api-access-pggxk\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.378706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.372010    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-credential-socket\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.378706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.372112    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-certs\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.378706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.372134    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfe33049-c39b-45f7-97c2-78dbd003c401-istiod-ca-cert\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.378706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.372258    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-socket\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.378706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.372283    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-envoy\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.378706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.372350    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-podinfo\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.473767 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.473719    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pggxk\" (UniqueName: \"kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-kube-api-access-pggxk\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.473979 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.473784    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-credential-socket\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.473979 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.473870    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-certs\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.473979 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.473894    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfe33049-c39b-45f7-97c2-78dbd003c401-istiod-ca-cert\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.474206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.474012    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-socket\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.474206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.474034    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-envoy\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.474206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.474115    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-podinfo\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.474374 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.474248    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-token\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.474374 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.474321    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-data\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.474374 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.474322    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-credential-socket\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.474721 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.474691    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfe33049-c39b-45f7-97c2-78dbd003c401-istiod-ca-cert\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.474938 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.474911    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-certs\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.475019 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.474970    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-socket\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.475019 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.475011    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-data\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.477989 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.477924    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-envoy\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.478162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.478051    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-podinfo\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.483869 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.483846    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pggxk\" (UniqueName: \"kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-kube-api-access-pggxk\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.486032 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.486007    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-token\") pod \"e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") " pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.693118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.686707    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:46.708764 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.700490    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:46.886217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:46.886071    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:47.079593 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:47.079561    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"]
Oct 30 19:06:47.083683 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:47.081989    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:06:47.083683 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:47.082074    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:06:47.083683 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:47.082141    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:06:47.096064 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:47.096037    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:47.666403 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:47.665449    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" event={"ID":"cfe33049-c39b-45f7-97c2-78dbd003c401","Type":"ContainerStarted","Data":"fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18"}
Oct 30 19:06:47.666403 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:47.665541    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" event={"ID":"cfe33049-c39b-45f7-97c2-78dbd003c401","Type":"ContainerStarted","Data":"fdd7978fb94732d9a04b9438883c3abb47e44cf037cfed32561f33ab7e6f3ecb"}
Oct 30 19:06:47.687310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:47.687262    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:47.698158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:47.697726    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" podStartSLOduration=1.697706766 podStartE2EDuration="1.697706766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:47.696128891 +0000 UTC m=+1829.412473965" watchObservedRunningTime="2026-10-30 19:06:47.697706766 +0000 UTC m=+1829.414051843"
Oct 30 19:06:47.850852 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:47.850762    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:48.554323 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:48.554296    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"
Oct 30 19:06:48.554729 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:48.554365    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"
Oct 30 19:06:48.707660 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:48.707620    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused" start-of-body=
Oct 30 19:06:48.707871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:48.707704    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" podUID="cfe33049-c39b-45f7-97c2-78dbd003c401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused"
Oct 30 19:06:49.028684 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:49.028326    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:49.687621 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:49.687582    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused" start-of-body=
Oct 30 19:06:49.688159 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:49.687654    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" podUID="cfe33049-c39b-45f7-97c2-78dbd003c401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused"
Oct 30 19:06:49.906534 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:49.906498    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:50.112208 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:50.111730    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:50.598167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:50.598125    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:50.693143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:50.689556    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused" start-of-body=
Oct 30 19:06:50.693143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:50.689632    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" podUID="cfe33049-c39b-45f7-97c2-78dbd003c401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused"
Oct 30 19:06:51.306123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:51.303654    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:51.640232 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:51.623607    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:51.645422 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:51.630218    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"]
Oct 30 19:06:51.688121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:51.684293    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"]
Oct 30 19:06:51.688121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:51.684415    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"
Oct 30 19:06:51.692267 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:51.689237    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused" start-of-body=
Oct 30 19:06:51.692267 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:51.689291    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" podUID="cfe33049-c39b-45f7-97c2-78dbd003c401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused"
Oct 30 19:06:51.785314 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:51.772034    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-d6f956\""
Oct 30 19:06:51.785314 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:51.774595    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwq9f\" (UniqueName: \"kubernetes.io/projected/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-kube-api-access-lwq9f\") pod \"payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t\" (UID: \"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"
Oct 30 19:06:51.785314 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:51.774636    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t\" (UID: \"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"
Oct 30 19:06:51.804936 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:51.804907    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-d6f956-dockercfg-9kfp5\""
Oct 30 19:06:52.062625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:51.992658    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:52.070235 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:52.051817    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwq9f\" (UniqueName: \"kubernetes.io/projected/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-kube-api-access-lwq9f\") pod \"payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t\" (UID: \"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"
Oct 30 19:06:52.070513 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:52.070486    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t\" (UID: \"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"
Oct 30 19:06:52.071846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:52.071817    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t\" (UID: \"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"
Oct 30 19:06:52.104132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:52.100065    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwq9f\" (UniqueName: \"kubernetes.io/projected/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-kube-api-access-lwq9f\") pod \"payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t\" (UID: \"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"
Oct 30 19:06:52.172665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:52.172589    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"
Oct 30 19:06:52.571173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:52.564694    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:52.612770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:52.611730    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"]
Oct 30 19:06:52.688793 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:52.688744    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused" start-of-body=
Oct 30 19:06:52.688939 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:52.688816    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" podUID="cfe33049-c39b-45f7-97c2-78dbd003c401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused"
Oct 30 19:06:53.011017 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.010980    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t" event={"ID":"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab","Type":"ContainerStarted","Data":"15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f"}
Oct 30 19:06:53.011495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.011043    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t" event={"ID":"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab","Type":"ContainerStarted","Data":"28fa77795b555172cad5c475d803aac60a7be0507c945fc36f03267d517813c4"}
Oct 30 19:06:53.011495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.011072    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"
Oct 30 19:06:53.031906 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.031856    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t" podStartSLOduration=2.031842373 podStartE2EDuration="2.031842373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:53.029214053 +0000 UTC m=+1834.745559163" watchObservedRunningTime="2026-10-30 19:06:53.031842373 +0000 UTC m=+1834.748187456"
Oct 30 19:06:53.597434 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.596432    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:53.604223 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.604171    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"]
Oct 30 19:06:53.688490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.687989    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused" start-of-body=
Oct 30 19:06:53.688490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.688047    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" podUID="cfe33049-c39b-45f7-97c2-78dbd003c401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused"
Oct 30 19:06:53.799420 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.799381    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:53.893564 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.893464    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7505-7a2b92"]
Oct 30 19:06:53.911669 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.911643    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-7a2b92"
Oct 30 19:06:53.921156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.919012    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-c64f8\""
Oct 30 19:06:53.922164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:53.921924    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7505-7a2b92"]
Oct 30 19:06:54.011006 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:54.010870    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljfn8\" (UniqueName: \"kubernetes.io/projected/e4054d98-bdde-4e45-b7da-f62f634ea96c-kube-api-access-ljfn8\") pod \"test-curl-7505-7a2b92\" (UID: \"e4054d98-bdde-4e45-b7da-f62f634ea96c\") " pod="opendatahub/test-curl-7505-7a2b92"
Oct 30 19:06:54.112954 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:54.112917    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljfn8\" (UniqueName: \"kubernetes.io/projected/e4054d98-bdde-4e45-b7da-f62f634ea96c-kube-api-access-ljfn8\") pod \"test-curl-7505-7a2b92\" (UID: \"e4054d98-bdde-4e45-b7da-f62f634ea96c\") " pod="opendatahub/test-curl-7505-7a2b92"
Oct 30 19:06:54.121961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:54.121938    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljfn8\" (UniqueName: \"kubernetes.io/projected/e4054d98-bdde-4e45-b7da-f62f634ea96c-kube-api-access-ljfn8\") pod \"test-curl-7505-7a2b92\" (UID: \"e4054d98-bdde-4e45-b7da-f62f634ea96c\") " pod="opendatahub/test-curl-7505-7a2b92"
Oct 30 19:06:54.238976 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:54.238947    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-7a2b92"
Oct 30 19:06:54.600238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:54.587417    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:54.695148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:54.694368    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7505-7a2b92"]
Oct 30 19:06:54.695148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:54.694439    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused" start-of-body=
Oct 30 19:06:54.695148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:54.694485    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" podUID="cfe33049-c39b-45f7-97c2-78dbd003c401" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.198:15021/healthz/ready\": dial tcp 10.132.0.198:15021: connect: connection refused"
Oct 30 19:06:54.696398 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:06:54.696374    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4054d98_bdde_4e45_b7da_f62f634ea96c.slice/crio-dae235685239653262002977017a7d74600acfb0d10bc31a4c168cc41c56c700 WatchSource:0}: Error finding container dae235685239653262002977017a7d74600acfb0d10bc31a4c168cc41c56c700: Status 404 returned error can't find the container with id dae235685239653262002977017a7d74600acfb0d10bc31a4c168cc41c56c700
Oct 30 19:06:55.044509 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.044462    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7505-7a2b92" event={"ID":"e4054d98-bdde-4e45-b7da-f62f634ea96c","Type":"ContainerStarted","Data":"0a77529c534cb4205fcb8af94f685c8f46f78446291f0938a03f2e66110f2513"}
Oct 30 19:06:55.044509 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.044513    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7505-7a2b92" event={"ID":"e4054d98-bdde-4e45-b7da-f62f634ea96c","Type":"ContainerStarted","Data":"dae235685239653262002977017a7d74600acfb0d10bc31a4c168cc41c56c700"}
Oct 30 19:06:55.074636 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.074571    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7505-7a2b92" podStartSLOduration=2.074551631 podStartE2EDuration="2.074551631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:55.068519618 +0000 UTC m=+1836.784864696" watchObservedRunningTime="2026-10-30 19:06:55.074551631 +0000 UTC m=+1836.790896708"
Oct 30 19:06:55.487121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.475470    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:55.759018 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.758937    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:55.759524 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.759497    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:55.760450 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.760427    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:06:55.796534 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.795021    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7505-7a2b92"]
Oct 30 19:06:55.801207 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.801172    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7505-7a2b92"]
Oct 30 19:06:55.951046 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.951014    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7505-297e1b"]
Oct 30 19:06:55.951851 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.951833    4925 cpu_manager.go:414] "containerMap: removing container" podUID="e4054d98-bdde-4e45-b7da-f62f634ea96c" containerName="test-curl-7505-7a2b92"
Oct 30 19:06:55.951914 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.951852    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4054d98-bdde-4e45-b7da-f62f634ea96c" containerName="test-curl-7505-7a2b92"
Oct 30 19:06:55.952000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.951991    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4054d98-bdde-4e45-b7da-f62f634ea96c" containerName="test-curl-7505-7a2b92"
Oct 30 19:06:55.956030 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.956000    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-297e1b"
Oct 30 19:06:55.970130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.959048    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmk9w\" (UniqueName: \"kubernetes.io/projected/fcee6050-6b2a-4c16-bc43-fc8d30a832ab-kube-api-access-fmk9w\") pod \"test-curl-7505-297e1b\" (UID: \"fcee6050-6b2a-4c16-bc43-fc8d30a832ab\") " pod="opendatahub/test-curl-7505-297e1b"
Oct 30 19:06:55.970130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:55.962811    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7505-297e1b"]
Oct 30 19:06:56.061653 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:56.061528    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmk9w\" (UniqueName: \"kubernetes.io/projected/fcee6050-6b2a-4c16-bc43-fc8d30a832ab-kube-api-access-fmk9w\") pod \"test-curl-7505-297e1b\" (UID: \"fcee6050-6b2a-4c16-bc43-fc8d30a832ab\") " pod="opendatahub/test-curl-7505-297e1b"
Oct 30 19:06:56.075550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:56.075517    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmk9w\" (UniqueName: \"kubernetes.io/projected/fcee6050-6b2a-4c16-bc43-fc8d30a832ab-kube-api-access-fmk9w\") pod \"test-curl-7505-297e1b\" (UID: \"fcee6050-6b2a-4c16-bc43-fc8d30a832ab\") " pod="opendatahub/test-curl-7505-297e1b"
Oct 30 19:06:56.277905 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:56.277868    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-297e1b"
Oct 30 19:06:56.385226 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:56.378424    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:56.662226 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:56.662195    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7505-297e1b"]
Oct 30 19:06:57.115687 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:57.114921    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:57.117215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:57.117182    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7505-297e1b" event={"ID":"fcee6050-6b2a-4c16-bc43-fc8d30a832ab","Type":"ContainerStarted","Data":"a9ca4b79c11154541baeb42cb3d42de1eb731d7978e65b126687ed833912e393"}
Oct 30 19:06:57.117409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:57.117389    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7505-297e1b" event={"ID":"fcee6050-6b2a-4c16-bc43-fc8d30a832ab","Type":"ContainerStarted","Data":"0294f3c01841bb81f02b0f4ffa9cbaeccb6a35e36168b5bd4be637758d15e554"}
Oct 30 19:06:57.117520 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:57.117416    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7505-7a2b92" podUID="e4054d98-bdde-4e45-b7da-f62f634ea96c" containerName="test-curl-7505-7a2b92" containerID="cri-o://0a77529c534cb4205fcb8af94f685c8f46f78446291f0938a03f2e66110f2513" gracePeriod=2
Oct 30 19:06:57.207971 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:57.207904    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7505-297e1b" podStartSLOduration=2.207884612 podStartE2EDuration="2.207884612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:57.175570623 +0000 UTC m=+1838.891915699" watchObservedRunningTime="2026-10-30 19:06:57.207884612 +0000 UTC m=+1838.924229689"
Oct 30 19:06:57.782118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:57.781786    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:57.850541 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:57.850495    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7505-297e1b"]
Oct 30 19:06:57.855045 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:57.855009    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7505-297e1b"]
Oct 30 19:06:58.005964 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.005924    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7505-1eea7e"]
Oct 30 19:06:58.007031 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.007001    4925 cpu_manager.go:414] "containerMap: removing container" podUID="fcee6050-6b2a-4c16-bc43-fc8d30a832ab" containerName="test-curl-7505-297e1b"
Oct 30 19:06:58.007031 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.007028    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fcee6050-6b2a-4c16-bc43-fc8d30a832ab" containerName="test-curl-7505-297e1b"
Oct 30 19:06:58.007264 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.007236    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="fcee6050-6b2a-4c16-bc43-fc8d30a832ab" containerName="test-curl-7505-297e1b"
Oct 30 19:06:58.011562 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.011383    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-1eea7e"
Oct 30 19:06:58.016397 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.016373    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7505-1eea7e"]
Oct 30 19:06:58.129821 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.129745    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drnb2\" (UniqueName: \"kubernetes.io/projected/6ff17919-6dc9-452b-bb57-3303ec0b46a3-kube-api-access-drnb2\") pod \"test-curl-7505-1eea7e\" (UID: \"6ff17919-6dc9-452b-bb57-3303ec0b46a3\") " pod="opendatahub/test-curl-7505-1eea7e"
Oct 30 19:06:58.231723 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.231692    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drnb2\" (UniqueName: \"kubernetes.io/projected/6ff17919-6dc9-452b-bb57-3303ec0b46a3-kube-api-access-drnb2\") pod \"test-curl-7505-1eea7e\" (UID: \"6ff17919-6dc9-452b-bb57-3303ec0b46a3\") " pod="opendatahub/test-curl-7505-1eea7e"
Oct 30 19:06:58.239575 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.239545    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drnb2\" (UniqueName: \"kubernetes.io/projected/6ff17919-6dc9-452b-bb57-3303ec0b46a3-kube-api-access-drnb2\") pod \"test-curl-7505-1eea7e\" (UID: \"6ff17919-6dc9-452b-bb57-3303ec0b46a3\") " pod="opendatahub/test-curl-7505-1eea7e"
Oct 30 19:06:58.325801 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.325770    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-1eea7e"
Oct 30 19:06:58.547778 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.533182    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:58.610835 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.610793    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7505-1eea7e"]
Oct 30 19:06:58.612725 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:06:58.612691    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ff17919_6dc9_452b_bb57_3303ec0b46a3.slice/crio-6f361b26d56b60fc162f0549176df8a7920a2b9a257bc62553b0370dbb95a561 WatchSource:0}: Error finding container 6f361b26d56b60fc162f0549176df8a7920a2b9a257bc62553b0370dbb95a561: Status 404 returned error can't find the container with id 6f361b26d56b60fc162f0549176df8a7920a2b9a257bc62553b0370dbb95a561
Oct 30 19:06:58.868880 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:58.868834    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz" podUID="a3ce381c-876f-4c34-8403-196b4eef627c" containerName="main" containerID="cri-o://eb89fae0c93e6b71e7d1cd5244ae8d41b1eacdb7a9bb9dbe1225dd8890601595" gracePeriod=45
Oct 30 19:06:59.136987 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.136844    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7505-1eea7e" event={"ID":"6ff17919-6dc9-452b-bb57-3303ec0b46a3","Type":"ContainerStarted","Data":"4b2c713d3b7f55c50a05cf755d9f74c748aad0ca0b6e412818266f48fa3cb7e6"}
Oct 30 19:06:59.136987 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.136892    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7505-1eea7e" event={"ID":"6ff17919-6dc9-452b-bb57-3303ec0b46a3","Type":"ContainerStarted","Data":"6f361b26d56b60fc162f0549176df8a7920a2b9a257bc62553b0370dbb95a561"}
Oct 30 19:06:59.142145 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.139317    4925 generic.go:356] "Generic (PLEG): container finished" podID="a3ce381c-876f-4c34-8403-196b4eef627c" containerID="eb89fae0c93e6b71e7d1cd5244ae8d41b1eacdb7a9bb9dbe1225dd8890601595" exitCode=0
Oct 30 19:06:59.142145 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.139357    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz" event={"ID":"a3ce381c-876f-4c34-8403-196b4eef627c","Type":"ContainerDied","Data":"eb89fae0c93e6b71e7d1cd5244ae8d41b1eacdb7a9bb9dbe1225dd8890601595"}
Oct 30 19:06:59.142145 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.139507    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7505-297e1b" podUID="fcee6050-6b2a-4c16-bc43-fc8d30a832ab" containerName="test-curl-7505-297e1b" containerID="cri-o://a9ca4b79c11154541baeb42cb3d42de1eb731d7978e65b126687ed833912e393" gracePeriod=2
Oct 30 19:06:59.165918 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.165888    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:06:59.190187 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.189939    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7505-1eea7e" podStartSLOduration=2.189922318 podStartE2EDuration="2.189922318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:59.156132243 +0000 UTC m=+1840.872477320" watchObservedRunningTime="2026-10-30 19:06:59.189922318 +0000 UTC m=+1840.906267396"
Oct 30 19:06:59.290086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.287670    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:59.290086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.288288    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-model-cache\") pod \"a3ce381c-876f-4c34-8403-196b4eef627c\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") "
Oct 30 19:06:59.290086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.288367    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-dshm\") pod \"a3ce381c-876f-4c34-8403-196b4eef627c\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") "
Oct 30 19:06:59.290086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.288477    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-home\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-home\") pod \"a3ce381c-876f-4c34-8403-196b4eef627c\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") "
Oct 30 19:06:59.290086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.288641    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3ce381c-876f-4c34-8403-196b4eef627c-kube-api-access-nq5qk\" (UniqueName: \"kubernetes.io/projected/a3ce381c-876f-4c34-8403-196b4eef627c-kube-api-access-nq5qk\") pod \"a3ce381c-876f-4c34-8403-196b4eef627c\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") "
Oct 30 19:06:59.290086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.288809    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-tmp-dir\") pod \"a3ce381c-876f-4c34-8403-196b4eef627c\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") "
Oct 30 19:06:59.290086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.288845    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a3ce381c-876f-4c34-8403-196b4eef627c-tls-certs\" (UniqueName: \"kubernetes.io/secret/a3ce381c-876f-4c34-8403-196b4eef627c-tls-certs\") pod \"a3ce381c-876f-4c34-8403-196b4eef627c\" (UID: \"a3ce381c-876f-4c34-8403-196b4eef627c\") "
Oct 30 19:06:59.311152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.310485    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-home" pod "a3ce381c-876f-4c34-8403-196b4eef627c" (UID: "a3ce381c-876f-4c34-8403-196b4eef627c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:59.314176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.312254    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-model-cache" pod "a3ce381c-876f-4c34-8403-196b4eef627c" (UID: "a3ce381c-876f-4c34-8403-196b4eef627c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:59.314176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.313155    4925 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-home\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:59.314176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.313187    4925 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-model-cache\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:59.314176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.313694    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-dshm" pod "a3ce381c-876f-4c34-8403-196b4eef627c" (UID: "a3ce381c-876f-4c34-8403-196b4eef627c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:59.314176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.313864    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-tmp-dir" pod "a3ce381c-876f-4c34-8403-196b4eef627c" (UID: "a3ce381c-876f-4c34-8403-196b4eef627c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:59.317325 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.315019    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3ce381c-876f-4c34-8403-196b4eef627c-tls-certs" pod "a3ce381c-876f-4c34-8403-196b4eef627c" (UID: "a3ce381c-876f-4c34-8403-196b4eef627c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:06:59.319004 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.318827    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3ce381c-876f-4c34-8403-196b4eef627c-kube-api-access-nq5qk" pod "a3ce381c-876f-4c34-8403-196b4eef627c" (UID: "a3ce381c-876f-4c34-8403-196b4eef627c"). InnerVolumeSpecName "kube-api-access-nq5qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:59.414723 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.414689    4925 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-tmp-dir\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:59.414723 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.414725    4925 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3ce381c-876f-4c34-8403-196b4eef627c-tls-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:59.414998 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.414739    4925 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3ce381c-876f-4c34-8403-196b4eef627c-dshm\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:59.414998 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.414757    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq5qk\" (UniqueName: \"kubernetes.io/projected/a3ce381c-876f-4c34-8403-196b4eef627c-kube-api-access-nq5qk\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:59.498637 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.498611    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-7a2b92"
Oct 30 19:06:59.617705 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.617629    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4054d98-bdde-4e45-b7da-f62f634ea96c-kube-api-access-ljfn8\" (UniqueName: \"kubernetes.io/projected/e4054d98-bdde-4e45-b7da-f62f634ea96c-kube-api-access-ljfn8\") pod \"e4054d98-bdde-4e45-b7da-f62f634ea96c\" (UID: \"e4054d98-bdde-4e45-b7da-f62f634ea96c\") "
Oct 30 19:06:59.619826 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.619801    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4054d98-bdde-4e45-b7da-f62f634ea96c-kube-api-access-ljfn8" pod "e4054d98-bdde-4e45-b7da-f62f634ea96c" (UID: "e4054d98-bdde-4e45-b7da-f62f634ea96c"). InnerVolumeSpecName "kube-api-access-ljfn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:59.719549 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.719519    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljfn8\" (UniqueName: \"kubernetes.io/projected/e4054d98-bdde-4e45-b7da-f62f634ea96c-kube-api-access-ljfn8\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:59.825172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.825142    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7505-1eea7e"]
Oct 30 19:06:59.831575 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.831549    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7505-1eea7e"]
Oct 30 19:06:59.991960 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:06:59.990498    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:00.045151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.044832    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7505-3d0374"]
Oct 30 19:07:00.046576 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.046543    4925 cpu_manager.go:414] "containerMap: removing container" podUID="6ff17919-6dc9-452b-bb57-3303ec0b46a3" containerName="test-curl-7505-1eea7e"
Oct 30 19:07:00.046760 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.046746    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ff17919-6dc9-452b-bb57-3303ec0b46a3" containerName="test-curl-7505-1eea7e"
Oct 30 19:07:00.046874 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.046864    4925 cpu_manager.go:414] "containerMap: removing container" podUID="a3ce381c-876f-4c34-8403-196b4eef627c" containerName="main"
Oct 30 19:07:00.046976 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.046965    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3ce381c-876f-4c34-8403-196b4eef627c" containerName="main"
Oct 30 19:07:00.048220 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.047293    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ff17919-6dc9-452b-bb57-3303ec0b46a3" containerName="test-curl-7505-1eea7e"
Oct 30 19:07:00.048451 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.048433    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3ce381c-876f-4c34-8403-196b4eef627c" containerName="main"
Oct 30 19:07:00.052746 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.052729    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7505-3d0374"]
Oct 30 19:07:00.052960 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.052947    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-3d0374"
Oct 30 19:07:00.126273 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.126247    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcslf\" (UniqueName: \"kubernetes.io/projected/f22771ca-4d1d-4959-90ce-40d7e1e07662-kube-api-access-mcslf\") pod \"test-curl-7505-3d0374\" (UID: \"f22771ca-4d1d-4959-90ce-40d7e1e07662\") " pod="opendatahub/test-curl-7505-3d0374"
Oct 30 19:07:00.150160 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.150127    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz" event={"ID":"a3ce381c-876f-4c34-8403-196b4eef627c","Type":"ContainerDied","Data":"2a388a1b7cac82a5ffe1dddda9e94c431ae2934ce9e1d5067d12b513d3b2cb64"}
Oct 30 19:07:00.150299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.150167    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"
Oct 30 19:07:00.150299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.150189    4925 scope.go:122] "RemoveContainer" containerID="eb89fae0c93e6b71e7d1cd5244ae8d41b1eacdb7a9bb9dbe1225dd8890601595"
Oct 30 19:07:00.151984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.151961    4925 generic.go:356] "Generic (PLEG): container finished" podID="e4054d98-bdde-4e45-b7da-f62f634ea96c" containerID="0a77529c534cb4205fcb8af94f685c8f46f78446291f0938a03f2e66110f2513" exitCode=137
Oct 30 19:07:00.152321 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.152007    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-7a2b92"
Oct 30 19:07:00.169919 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:00.169891    4925 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7505-7a2b92\" is forbidden: User \"system:node:ip-10-0-134-58.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-58.us-east-2.compute.internal' and this object" podUID="e4054d98-bdde-4e45-b7da-f62f634ea96c" pod="opendatahub/test-curl-7505-7a2b92"
Oct 30 19:07:00.172531 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:00.172198    4925 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7505-7a2b92\" is forbidden: User \"system:node:ip-10-0-134-58.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-58.us-east-2.compute.internal' and this object" podUID="e4054d98-bdde-4e45-b7da-f62f634ea96c" pod="opendatahub/test-curl-7505-7a2b92"
Oct 30 19:07:00.172656 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.172257    4925 scope.go:122] "RemoveContainer" containerID="0a77529c534cb4205fcb8af94f685c8f46f78446291f0938a03f2e66110f2513"
Oct 30 19:07:00.186169 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.186086    4925 scope.go:122] "RemoveContainer" containerID="0a77529c534cb4205fcb8af94f685c8f46f78446291f0938a03f2e66110f2513"
Oct 30 19:07:00.187468 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:00.187179    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a77529c534cb4205fcb8af94f685c8f46f78446291f0938a03f2e66110f2513\": container with ID starting with 0a77529c534cb4205fcb8af94f685c8f46f78446291f0938a03f2e66110f2513 not found: ID does not exist" containerID="0a77529c534cb4205fcb8af94f685c8f46f78446291f0938a03f2e66110f2513"
Oct 30 19:07:00.187468 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.187215    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a77529c534cb4205fcb8af94f685c8f46f78446291f0938a03f2e66110f2513"} err="failed to get container status \"0a77529c534cb4205fcb8af94f685c8f46f78446291f0938a03f2e66110f2513\": rpc error: code = NotFound desc = could not find container \"0a77529c534cb4205fcb8af94f685c8f46f78446291f0938a03f2e66110f2513\": container with ID starting with 0a77529c534cb4205fcb8af94f685c8f46f78446291f0938a03f2e66110f2513 not found: ID does not exist"
Oct 30 19:07:00.195062 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.194125    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"]
Oct 30 19:07:00.199292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.199271    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-ef6be7/auth-test-model-ef6be7-kserve-5dfc6bc5c7-z9bxz"]
Oct 30 19:07:00.227735 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.227706    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcslf\" (UniqueName: \"kubernetes.io/projected/f22771ca-4d1d-4959-90ce-40d7e1e07662-kube-api-access-mcslf\") pod \"test-curl-7505-3d0374\" (UID: \"f22771ca-4d1d-4959-90ce-40d7e1e07662\") " pod="opendatahub/test-curl-7505-3d0374"
Oct 30 19:07:00.236573 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.236549    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcslf\" (UniqueName: \"kubernetes.io/projected/f22771ca-4d1d-4959-90ce-40d7e1e07662-kube-api-access-mcslf\") pod \"test-curl-7505-3d0374\" (UID: \"f22771ca-4d1d-4959-90ce-40d7e1e07662\") " pod="opendatahub/test-curl-7505-3d0374"
Oct 30 19:07:00.374146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.374108    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-3d0374"
Oct 30 19:07:00.426594 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.426545    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3ce381c-876f-4c34-8403-196b4eef627c" path="/var/lib/kubelet/pods/a3ce381c-876f-4c34-8403-196b4eef627c/volumes"
Oct 30 19:07:00.427070 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.427019    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4054d98-bdde-4e45-b7da-f62f634ea96c" path="/var/lib/kubelet/pods/e4054d98-bdde-4e45-b7da-f62f634ea96c/volumes"
Oct 30 19:07:00.691922 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.691833    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:00.743930 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:00.737888    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7505-3d0374"]
Oct 30 19:07:01.165240 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:01.165163    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7505-3d0374" event={"ID":"f22771ca-4d1d-4959-90ce-40d7e1e07662","Type":"ContainerStarted","Data":"dc97ba46a3dfada1616dfc144269e07fa4bb967b389cb59e14b9acaebe2401dd"}
Oct 30 19:07:01.165240 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:01.165215    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7505-3d0374" event={"ID":"f22771ca-4d1d-4959-90ce-40d7e1e07662","Type":"ContainerStarted","Data":"9fc37b8288b5121424fa6154f0b3fd5ac07f0997b8c25b21587bf008e68f09e0"}
Oct 30 19:07:01.165829 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:01.165252    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7505-1eea7e" podUID="6ff17919-6dc9-452b-bb57-3303ec0b46a3" containerName="test-curl-7505-1eea7e" containerID="cri-o://4b2c713d3b7f55c50a05cf755d9f74c748aad0ca0b6e412818266f48fa3cb7e6" gracePeriod=2
Oct 30 19:07:01.182762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:01.182709    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7505-3d0374" podStartSLOduration=2.182693469 podStartE2EDuration="2.182693469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:01.179014576 +0000 UTC m=+1842.895359653" watchObservedRunningTime="2026-10-30 19:07:01.182693469 +0000 UTC m=+1842.899038547"
Oct 30 19:07:01.335004 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:01.334976    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-297e1b"
Oct 30 19:07:01.461121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:01.449826    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcee6050-6b2a-4c16-bc43-fc8d30a832ab-kube-api-access-fmk9w\" (UniqueName: \"kubernetes.io/projected/fcee6050-6b2a-4c16-bc43-fc8d30a832ab-kube-api-access-fmk9w\") pod \"fcee6050-6b2a-4c16-bc43-fc8d30a832ab\" (UID: \"fcee6050-6b2a-4c16-bc43-fc8d30a832ab\") "
Oct 30 19:07:01.484153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:01.481335    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:01.509368 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:01.509309    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcee6050-6b2a-4c16-bc43-fc8d30a832ab-kube-api-access-fmk9w" pod "fcee6050-6b2a-4c16-bc43-fc8d30a832ab" (UID: "fcee6050-6b2a-4c16-bc43-fc8d30a832ab"). InnerVolumeSpecName "kube-api-access-fmk9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:01.553260 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:01.553226    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmk9w\" (UniqueName: \"kubernetes.io/projected/fcee6050-6b2a-4c16-bc43-fc8d30a832ab-kube-api-access-fmk9w\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:01.979968 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:01.979934    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7505-3d0374"]
Oct 30 19:07:01.987637 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:01.987606    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7505-3d0374"]
Oct 30 19:07:02.156860 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:02.156291    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:02.181002 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:02.180969    4925 generic.go:356] "Generic (PLEG): container finished" podID="fcee6050-6b2a-4c16-bc43-fc8d30a832ab" containerID="a9ca4b79c11154541baeb42cb3d42de1eb731d7978e65b126687ed833912e393" exitCode=137
Oct 30 19:07:02.181895 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:02.181880    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-297e1b"
Oct 30 19:07:02.182384 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:02.182369    4925 scope.go:122] "RemoveContainer" containerID="a9ca4b79c11154541baeb42cb3d42de1eb731d7978e65b126687ed833912e393"
Oct 30 19:07:02.193910 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:02.193886    4925 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7505-297e1b\" is forbidden: User \"system:node:ip-10-0-134-58.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-58.us-east-2.compute.internal' and this object" podUID="fcee6050-6b2a-4c16-bc43-fc8d30a832ab" pod="opendatahub/test-curl-7505-297e1b"
Oct 30 19:07:02.196310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:02.196293    4925 scope.go:122] "RemoveContainer" containerID="a9ca4b79c11154541baeb42cb3d42de1eb731d7978e65b126687ed833912e393"
Oct 30 19:07:02.196671 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:02.196642    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9ca4b79c11154541baeb42cb3d42de1eb731d7978e65b126687ed833912e393\": container with ID starting with a9ca4b79c11154541baeb42cb3d42de1eb731d7978e65b126687ed833912e393 not found: ID does not exist" containerID="a9ca4b79c11154541baeb42cb3d42de1eb731d7978e65b126687ed833912e393"
Oct 30 19:07:02.196792 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:02.196678    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9ca4b79c11154541baeb42cb3d42de1eb731d7978e65b126687ed833912e393"} err="failed to get container status \"a9ca4b79c11154541baeb42cb3d42de1eb731d7978e65b126687ed833912e393\": rpc error: code = NotFound desc = could not find container \"a9ca4b79c11154541baeb42cb3d42de1eb731d7978e65b126687ed833912e393\": container with ID starting with a9ca4b79c11154541baeb42cb3d42de1eb731d7978e65b126687ed833912e393 not found: ID does not exist"
Oct 30 19:07:02.426588 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:02.426559    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fcee6050-6b2a-4c16-bc43-fc8d30a832ab" path="/var/lib/kubelet/pods/fcee6050-6b2a-4c16-bc43-fc8d30a832ab/volumes"
Oct 30 19:07:03.035183 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:03.035149    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:03.193844 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:03.193814    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7505-3d0374" podUID="f22771ca-4d1d-4959-90ce-40d7e1e07662" containerName="test-curl-7505-3d0374" containerID="cri-o://dc97ba46a3dfada1616dfc144269e07fa4bb967b389cb59e14b9acaebe2401dd" gracePeriod=2
Oct 30 19:07:03.362567 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:03.362542    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-1eea7e"
Oct 30 19:07:03.494847 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:03.494815    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ff17919-6dc9-452b-bb57-3303ec0b46a3-kube-api-access-drnb2\" (UniqueName: \"kubernetes.io/projected/6ff17919-6dc9-452b-bb57-3303ec0b46a3-kube-api-access-drnb2\") pod \"6ff17919-6dc9-452b-bb57-3303ec0b46a3\" (UID: \"6ff17919-6dc9-452b-bb57-3303ec0b46a3\") "
Oct 30 19:07:03.497324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:03.497293    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ff17919-6dc9-452b-bb57-3303ec0b46a3-kube-api-access-drnb2" pod "6ff17919-6dc9-452b-bb57-3303ec0b46a3" (UID: "6ff17919-6dc9-452b-bb57-3303ec0b46a3"). InnerVolumeSpecName "kube-api-access-drnb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:03.647192 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:03.638470    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drnb2\" (UniqueName: \"kubernetes.io/projected/6ff17919-6dc9-452b-bb57-3303ec0b46a3-kube-api-access-drnb2\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:03.809201 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:03.763181    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:04.037664 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:04.037629    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"
Oct 30 19:07:04.089572 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:04.089390    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:04.269134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:04.260546    4925 generic.go:356] "Generic (PLEG): container finished" podID="6ff17919-6dc9-452b-bb57-3303ec0b46a3" containerID="4b2c713d3b7f55c50a05cf755d9f74c748aad0ca0b6e412818266f48fa3cb7e6" exitCode=137
Oct 30 19:07:04.269134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:04.260795    4925 scope.go:122] "RemoveContainer" containerID="4b2c713d3b7f55c50a05cf755d9f74c748aad0ca0b6e412818266f48fa3cb7e6"
Oct 30 19:07:04.269134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:04.261019    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-1eea7e"
Oct 30 19:07:04.314119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:04.298205    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:04.350483 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:04.350434    4925 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7505-1eea7e\" is forbidden: User \"system:node:ip-10-0-134-58.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-58.us-east-2.compute.internal' and this object" podUID="6ff17919-6dc9-452b-bb57-3303ec0b46a3" pod="opendatahub/test-curl-7505-1eea7e"
Oct 30 19:07:04.372579 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:04.372439    4925 scope.go:122] "RemoveContainer" containerID="4b2c713d3b7f55c50a05cf755d9f74c748aad0ca0b6e412818266f48fa3cb7e6"
Oct 30 19:07:04.372953 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:04.372915    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b2c713d3b7f55c50a05cf755d9f74c748aad0ca0b6e412818266f48fa3cb7e6\": container with ID starting with 4b2c713d3b7f55c50a05cf755d9f74c748aad0ca0b6e412818266f48fa3cb7e6 not found: ID does not exist" containerID="4b2c713d3b7f55c50a05cf755d9f74c748aad0ca0b6e412818266f48fa3cb7e6"
Oct 30 19:07:04.373083 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:04.372981    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b2c713d3b7f55c50a05cf755d9f74c748aad0ca0b6e412818266f48fa3cb7e6"} err="failed to get container status \"4b2c713d3b7f55c50a05cf755d9f74c748aad0ca0b6e412818266f48fa3cb7e6\": rpc error: code = NotFound desc = could not find container \"4b2c713d3b7f55c50a05cf755d9f74c748aad0ca0b6e412818266f48fa3cb7e6\": container with ID starting with 4b2c713d3b7f55c50a05cf755d9f74c748aad0ca0b6e412818266f48fa3cb7e6 not found: ID does not exist"
Oct 30 19:07:04.495160 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:04.485606    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ff17919-6dc9-452b-bb57-3303ec0b46a3" path="/var/lib/kubelet/pods/6ff17919-6dc9-452b-bb57-3303ec0b46a3/volumes"
Oct 30 19:07:04.910082 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:04.910047    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:05.105194 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:05.103852    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:05.270870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:05.270841    4925 generic.go:356] "Generic (PLEG): container finished" podID="f22771ca-4d1d-4959-90ce-40d7e1e07662" containerID="dc97ba46a3dfada1616dfc144269e07fa4bb967b389cb59e14b9acaebe2401dd" exitCode=137
Oct 30 19:07:05.430864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:05.430837    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-3d0374"
Oct 30 19:07:05.610184 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:05.606744    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f22771ca-4d1d-4959-90ce-40d7e1e07662-kube-api-access-mcslf\" (UniqueName: \"kubernetes.io/projected/f22771ca-4d1d-4959-90ce-40d7e1e07662-kube-api-access-mcslf\") pod \"f22771ca-4d1d-4959-90ce-40d7e1e07662\" (UID: \"f22771ca-4d1d-4959-90ce-40d7e1e07662\") "
Oct 30 19:07:05.648121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:05.643329    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f22771ca-4d1d-4959-90ce-40d7e1e07662-kube-api-access-mcslf" pod "f22771ca-4d1d-4959-90ce-40d7e1e07662" (UID: "f22771ca-4d1d-4959-90ce-40d7e1e07662"). InnerVolumeSpecName "kube-api-access-mcslf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:05.719172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:05.719133    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcslf\" (UniqueName: \"kubernetes.io/projected/f22771ca-4d1d-4959-90ce-40d7e1e07662-kube-api-access-mcslf\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:06.049119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:06.048365    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:06.326929 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:06.326846    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:06.358636 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:06.358438    4925 scope.go:122] "RemoveContainer" containerID="dc97ba46a3dfada1616dfc144269e07fa4bb967b389cb59e14b9acaebe2401dd"
Oct 30 19:07:06.358862 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:06.358649    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7505-3d0374"
Oct 30 19:07:06.370756 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:06.370725    4925 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7505-3d0374\" is forbidden: User \"system:node:ip-10-0-134-58.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-58.us-east-2.compute.internal' and this object" podUID="f22771ca-4d1d-4959-90ce-40d7e1e07662" pod="opendatahub/test-curl-7505-3d0374"
Oct 30 19:07:06.424855 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:06.424827    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f22771ca-4d1d-4959-90ce-40d7e1e07662" path="/var/lib/kubelet/pods/f22771ca-4d1d-4959-90ce-40d7e1e07662/volumes"
Oct 30 19:07:06.756650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:06.756606    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:07.138233 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:07.138140    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:08.098392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:08.087575    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:08.342986 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:08.342948    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:08.537811 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:08.537777    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:08.643467 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:08.643433    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb" podUID="b920c3b4-6749-4ae3-895f-5de002df2777" containerName="main" containerID="cri-o://420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a" gracePeriod=45
Oct 30 19:07:08.952357 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:08.952292    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:09.069323 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.069298    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:07:09.147017 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.146928    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-dshm\") pod \"b920c3b4-6749-4ae3-895f-5de002df2777\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") "
Oct 30 19:07:09.147489 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.147404    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-tmp-dir" pod "b920c3b4-6749-4ae3-895f-5de002df2777" (UID: "b920c3b4-6749-4ae3-895f-5de002df2777"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:09.147489 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.147048    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-tmp-dir\") pod \"b920c3b4-6749-4ae3-895f-5de002df2777\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") "
Oct 30 19:07:09.147608 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.147503    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-model-cache\") pod \"b920c3b4-6749-4ae3-895f-5de002df2777\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") "
Oct 30 19:07:09.147773 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.147758    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-model-cache" pod "b920c3b4-6749-4ae3-895f-5de002df2777" (UID: "b920c3b4-6749-4ae3-895f-5de002df2777"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:09.148315 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.147949    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b920c3b4-6749-4ae3-895f-5de002df2777-tls-certs\" (UniqueName: \"kubernetes.io/secret/b920c3b4-6749-4ae3-895f-5de002df2777-tls-certs\") pod \"b920c3b4-6749-4ae3-895f-5de002df2777\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") "
Oct 30 19:07:09.148767 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.148741    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-home" pod "b920c3b4-6749-4ae3-895f-5de002df2777" (UID: "b920c3b4-6749-4ae3-895f-5de002df2777"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:09.148888 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.148834    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-home\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-home\") pod \"b920c3b4-6749-4ae3-895f-5de002df2777\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") "
Oct 30 19:07:09.148964 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.148901    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b920c3b4-6749-4ae3-895f-5de002df2777-kube-api-access-txfrc\" (UniqueName: \"kubernetes.io/projected/b920c3b4-6749-4ae3-895f-5de002df2777-kube-api-access-txfrc\") pod \"b920c3b4-6749-4ae3-895f-5de002df2777\" (UID: \"b920c3b4-6749-4ae3-895f-5de002df2777\") "
Oct 30 19:07:09.149825 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.149803    4925 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-tmp-dir\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:09.149924 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.149829    4925 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-model-cache\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:09.151572 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.151545    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b920c3b4-6749-4ae3-895f-5de002df2777-tls-certs" pod "b920c3b4-6749-4ae3-895f-5de002df2777" (UID: "b920c3b4-6749-4ae3-895f-5de002df2777"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:07:09.158175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.157278    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b920c3b4-6749-4ae3-895f-5de002df2777-kube-api-access-txfrc" pod "b920c3b4-6749-4ae3-895f-5de002df2777" (UID: "b920c3b4-6749-4ae3-895f-5de002df2777"). InnerVolumeSpecName "kube-api-access-txfrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:09.158651 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.158626    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-dshm" pod "b920c3b4-6749-4ae3-895f-5de002df2777" (UID: "b920c3b4-6749-4ae3-895f-5de002df2777"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:09.251987 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.251899    4925 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-dshm\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:09.251987 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.251933    4925 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b920c3b4-6749-4ae3-895f-5de002df2777-tls-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:09.251987 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.251946    4925 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b920c3b4-6749-4ae3-895f-5de002df2777-home\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:09.251987 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.251959    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txfrc\" (UniqueName: \"kubernetes.io/projected/b920c3b4-6749-4ae3-895f-5de002df2777-kube-api-access-txfrc\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:09.387715 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.387683    4925 generic.go:356] "Generic (PLEG): container finished" podID="b920c3b4-6749-4ae3-895f-5de002df2777" containerID="420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a" exitCode=0
Oct 30 19:07:09.387906 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.387754    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"
Oct 30 19:07:09.387906 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.387769    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb" event={"ID":"b920c3b4-6749-4ae3-895f-5de002df2777","Type":"ContainerDied","Data":"420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a"}
Oct 30 19:07:09.387906 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.387803    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb" event={"ID":"b920c3b4-6749-4ae3-895f-5de002df2777","Type":"ContainerDied","Data":"62fe2151cdb9bb393de41d86501cf281b107f1fe57ebf26f28901a9e0841d567"}
Oct 30 19:07:09.387906 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.387822    4925 scope.go:122] "RemoveContainer" containerID="420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a"
Oct 30 19:07:09.405142 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.405084    4925 scope.go:122] "RemoveContainer" containerID="420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a"
Oct 30 19:07:09.405415 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:09.405394    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a\": container with ID starting with 420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a not found: ID does not exist" containerID="420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a"
Oct 30 19:07:09.405486 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.405426    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a"} err="failed to get container status \"420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a\": rpc error: code = NotFound desc = could not find container \"420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a\": container with ID starting with 420194d2f742e757a70eb817a72eb66fd731f363a6de23b884dd1dba83b2a00a not found: ID does not exist"
Oct 30 19:07:09.416440 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.416416    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"]
Oct 30 19:07:09.422334 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.422314    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w4-ed012e/auth-test-model-ed012e-kserve-568d54c5c-k74vb"]
Oct 30 19:07:09.591209 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.591174    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"]
Oct 30 19:07:09.592441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.592421    4925 cpu_manager.go:414] "containerMap: removing container" podUID="f22771ca-4d1d-4959-90ce-40d7e1e07662" containerName="test-curl-7505-3d0374"
Oct 30 19:07:09.592509 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.592445    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f22771ca-4d1d-4959-90ce-40d7e1e07662" containerName="test-curl-7505-3d0374"
Oct 30 19:07:09.592509 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.592503    4925 cpu_manager.go:414] "containerMap: removing container" podUID="b920c3b4-6749-4ae3-895f-5de002df2777" containerName="main"
Oct 30 19:07:09.592579 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.592512    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b920c3b4-6749-4ae3-895f-5de002df2777" containerName="main"
Oct 30 19:07:09.592733 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.592720    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="b920c3b4-6749-4ae3-895f-5de002df2777" containerName="main"
Oct 30 19:07:09.592767 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.592750    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="f22771ca-4d1d-4959-90ce-40d7e1e07662" containerName="test-curl-7505-3d0374"
Oct 30 19:07:09.598602 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.598575    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"
Oct 30 19:07:09.606272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.603221    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-1dd09d-dockercfg-4dl2n\""
Oct 30 19:07:09.606272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.603226    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-1dd09d\""
Oct 30 19:07:09.606272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.603905    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"]
Oct 30 19:07:09.622176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.622146    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:09.644772 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.644673    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"]
Oct 30 19:07:09.649480 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.649415    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"
Oct 30 19:07:09.660771 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.659327    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13c22a74-d32d-4400-bde2-eb77c344c8b9-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m\" (UID: \"13c22a74-d32d-4400-bde2-eb77c344c8b9\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"
Oct 30 19:07:09.660771 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.659410    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj48m\" (UniqueName: \"kubernetes.io/projected/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-kube-api-access-nj48m\") pod \"payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb\" (UID: \"cfe122bd-8ed6-4a29-8931-8b93ef12f26b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"
Oct 30 19:07:09.661693 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.661665    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks9kj\" (UniqueName: \"kubernetes.io/projected/13c22a74-d32d-4400-bde2-eb77c344c8b9-kube-api-access-ks9kj\") pod \"payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m\" (UID: \"13c22a74-d32d-4400-bde2-eb77c344c8b9\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"
Oct 30 19:07:09.661841 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.661750    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb\" (UID: \"cfe122bd-8ed6-4a29-8931-8b93ef12f26b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"
Oct 30 19:07:09.664783 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.664753    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"]
Oct 30 19:07:09.938762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.935804    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13c22a74-d32d-4400-bde2-eb77c344c8b9-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m\" (UID: \"13c22a74-d32d-4400-bde2-eb77c344c8b9\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"
Oct 30 19:07:09.938762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.935874    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj48m\" (UniqueName: \"kubernetes.io/projected/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-kube-api-access-nj48m\") pod \"payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb\" (UID: \"cfe122bd-8ed6-4a29-8931-8b93ef12f26b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"
Oct 30 19:07:09.938762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.936081    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks9kj\" (UniqueName: \"kubernetes.io/projected/13c22a74-d32d-4400-bde2-eb77c344c8b9-kube-api-access-ks9kj\") pod \"payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m\" (UID: \"13c22a74-d32d-4400-bde2-eb77c344c8b9\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"
Oct 30 19:07:09.938762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.936194    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb\" (UID: \"cfe122bd-8ed6-4a29-8931-8b93ef12f26b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"
Oct 30 19:07:09.938762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.937315    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb\" (UID: \"cfe122bd-8ed6-4a29-8931-8b93ef12f26b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"
Oct 30 19:07:09.938762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.938213    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13c22a74-d32d-4400-bde2-eb77c344c8b9-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m\" (UID: \"13c22a74-d32d-4400-bde2-eb77c344c8b9\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"
Oct 30 19:07:10.029195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:09.979563    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:10.029195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:10.008846    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks9kj\" (UniqueName: \"kubernetes.io/projected/13c22a74-d32d-4400-bde2-eb77c344c8b9-kube-api-access-ks9kj\") pod \"payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m\" (UID: \"13c22a74-d32d-4400-bde2-eb77c344c8b9\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"
Oct 30 19:07:10.034698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:10.034668    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj48m\" (UniqueName: \"kubernetes.io/projected/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-kube-api-access-nj48m\") pod \"payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb\" (UID: \"cfe122bd-8ed6-4a29-8931-8b93ef12f26b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"
Oct 30 19:07:10.225213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:10.221738    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:10.225213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:10.221973    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"
Oct 30 19:07:10.282114 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:10.270164    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"
Oct 30 19:07:10.441819 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:10.441783    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:10.444301 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:10.444266    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b920c3b4-6749-4ae3-895f-5de002df2777" path="/var/lib/kubelet/pods/b920c3b4-6749-4ae3-895f-5de002df2777/volumes"
Oct 30 19:07:10.579352 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:10.579322    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"]
Oct 30 19:07:10.579894 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:07:10.579839    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfe122bd_8ed6_4a29_8931_8b93ef12f26b.slice/crio-ab67541cce7a925b26bc775080c103c31acf1e166aa1dc77af19a85416efffc9 WatchSource:0}: Error finding container ab67541cce7a925b26bc775080c103c31acf1e166aa1dc77af19a85416efffc9: Status 404 returned error can't find the container with id ab67541cce7a925b26bc775080c103c31acf1e166aa1dc77af19a85416efffc9
Oct 30 19:07:10.619774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:10.619736    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"]
Oct 30 19:07:10.620563 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:07:10.620540    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c22a74_d32d_4400_bde2_eb77c344c8b9.slice/crio-3836e3c1dcfb65d1b01e2bb1f52f42f00b04e8c1e27835873a7adce58c617a85 WatchSource:0}: Error finding container 3836e3c1dcfb65d1b01e2bb1f52f42f00b04e8c1e27835873a7adce58c617a85: Status 404 returned error can't find the container with id 3836e3c1dcfb65d1b01e2bb1f52f42f00b04e8c1e27835873a7adce58c617a85
Oct 30 19:07:10.639140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:10.639114    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:10.932740 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:10.932706    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:11.128580 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.128545    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:11.312987 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.312887    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:11.412217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.412179    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb" event={"ID":"cfe122bd-8ed6-4a29-8931-8b93ef12f26b","Type":"ContainerStarted","Data":"1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596"}
Oct 30 19:07:11.412423 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.412239    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb" event={"ID":"cfe122bd-8ed6-4a29-8931-8b93ef12f26b","Type":"ContainerStarted","Data":"ab67541cce7a925b26bc775080c103c31acf1e166aa1dc77af19a85416efffc9"}
Oct 30 19:07:11.412423 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.412265    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"
Oct 30 19:07:11.414353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.414326    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m" event={"ID":"13c22a74-d32d-4400-bde2-eb77c344c8b9","Type":"ContainerStarted","Data":"7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2"}
Oct 30 19:07:11.414495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.414365    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m" event={"ID":"13c22a74-d32d-4400-bde2-eb77c344c8b9","Type":"ContainerStarted","Data":"3836e3c1dcfb65d1b01e2bb1f52f42f00b04e8c1e27835873a7adce58c617a85"}
Oct 30 19:07:11.414495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.414469    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"
Oct 30 19:07:11.428441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.428384    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb" podStartSLOduration=2.428367465 podStartE2EDuration="2.428367465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:11.427256918 +0000 UTC m=+1853.143602006" watchObservedRunningTime="2026-10-30 19:07:11.428367465 +0000 UTC m=+1853.144712543"
Oct 30 19:07:11.453569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.452922    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m" podStartSLOduration=2.452898381 podStartE2EDuration="2.452898381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:11.443972028 +0000 UTC m=+1853.160317102" watchObservedRunningTime="2026-10-30 19:07:11.452898381 +0000 UTC m=+1853.169243457"
Oct 30 19:07:11.629051 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.628971    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"]
Oct 30 19:07:11.635022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.634993    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"
Oct 30 19:07:11.645542 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.645516    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"]
Oct 30 19:07:11.667693 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.667439    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-f3d6a5\""
Oct 30 19:07:11.667693 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.667488    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"]
Oct 30 19:07:11.677320 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.677203    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:11.679141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.679086    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"]
Oct 30 19:07:11.679359 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.679342    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"
Oct 30 19:07:11.776134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.775498    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-f3d6a5-dockercfg-6gzk5\""
Oct 30 19:07:11.916120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:11.912691    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:12.054991 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.054946    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57195712-20ec-4a9b-97ce-bfc531287e3a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv\" (UID: \"57195712-20ec-4a9b-97ce-bfc531287e3a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"
Oct 30 19:07:12.055384 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.055352    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d227w\" (UniqueName: \"kubernetes.io/projected/57195712-20ec-4a9b-97ce-bfc531287e3a-kube-api-access-d227w\") pod \"payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv\" (UID: \"57195712-20ec-4a9b-97ce-bfc531287e3a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"
Oct 30 19:07:12.055612 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.055594    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw\" (UID: \"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"
Oct 30 19:07:12.055736 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.055722    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c8bj\" (UniqueName: \"kubernetes.io/projected/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-kube-api-access-4c8bj\") pod \"payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw\" (UID: \"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"
Oct 30 19:07:12.157197 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.157159    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57195712-20ec-4a9b-97ce-bfc531287e3a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv\" (UID: \"57195712-20ec-4a9b-97ce-bfc531287e3a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"
Oct 30 19:07:12.157451 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.157316    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d227w\" (UniqueName: \"kubernetes.io/projected/57195712-20ec-4a9b-97ce-bfc531287e3a-kube-api-access-d227w\") pod \"payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv\" (UID: \"57195712-20ec-4a9b-97ce-bfc531287e3a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"
Oct 30 19:07:12.157550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.157458    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw\" (UID: \"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"
Oct 30 19:07:12.157550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.157499    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c8bj\" (UniqueName: \"kubernetes.io/projected/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-kube-api-access-4c8bj\") pod \"payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw\" (UID: \"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"
Oct 30 19:07:12.158810 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.158778    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw\" (UID: \"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"
Oct 30 19:07:12.160246 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.160223    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57195712-20ec-4a9b-97ce-bfc531287e3a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv\" (UID: \"57195712-20ec-4a9b-97ce-bfc531287e3a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"
Oct 30 19:07:12.169415 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.169342    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d227w\" (UniqueName: \"kubernetes.io/projected/57195712-20ec-4a9b-97ce-bfc531287e3a-kube-api-access-d227w\") pod \"payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv\" (UID: \"57195712-20ec-4a9b-97ce-bfc531287e3a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"
Oct 30 19:07:12.174428 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.174402    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c8bj\" (UniqueName: \"kubernetes.io/projected/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-kube-api-access-4c8bj\") pod \"payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw\" (UID: \"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"
Oct 30 19:07:12.257031 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.257001    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"
Oct 30 19:07:12.312462 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.312417    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"
Oct 30 19:07:12.375403 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.374954    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:12.592630 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:07:12.592564    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5fb074d_ef9f_44e0_92bd_5e4ab059b78e.slice/crio-290bb0a56a753958265b2275cc2e74475a6dfe2941141fbc5cfab5147730bde1 WatchSource:0}: Error finding container 290bb0a56a753958265b2275cc2e74475a6dfe2941141fbc5cfab5147730bde1: Status 404 returned error can't find the container with id 290bb0a56a753958265b2275cc2e74475a6dfe2941141fbc5cfab5147730bde1
Oct 30 19:07:12.593072 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.593045    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"]
Oct 30 19:07:12.735979 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:12.735950    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"]
Oct 30 19:07:12.736791 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:07:12.736763    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57195712_20ec_4a9b_97ce_bfc531287e3a.slice/crio-540395b15a79552cbe510f194f7f4b774259d5fdd671179702aca0f5ef9ef646 WatchSource:0}: Error finding container 540395b15a79552cbe510f194f7f4b774259d5fdd671179702aca0f5ef9ef646: Status 404 returned error can't find the container with id 540395b15a79552cbe510f194f7f4b774259d5fdd671179702aca0f5ef9ef646
Oct 30 19:07:13.113560 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.113470    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:13.304834 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:13.304799    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:07:13.474384 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.474349    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw" event={"ID":"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e","Type":"ContainerStarted","Data":"3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63"}
Oct 30 19:07:13.474867 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.474409    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw" event={"ID":"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e","Type":"ContainerStarted","Data":"290bb0a56a753958265b2275cc2e74475a6dfe2941141fbc5cfab5147730bde1"}
Oct 30 19:07:13.474867 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.474602    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"
Oct 30 19:07:13.477844 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.477506    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv" event={"ID":"57195712-20ec-4a9b-97ce-bfc531287e3a","Type":"ContainerStarted","Data":"07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4"}
Oct 30 19:07:13.477844 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.477558    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv" event={"ID":"57195712-20ec-4a9b-97ce-bfc531287e3a","Type":"ContainerStarted","Data":"540395b15a79552cbe510f194f7f4b774259d5fdd671179702aca0f5ef9ef646"}
Oct 30 19:07:13.478237 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.478213    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"
Oct 30 19:07:13.497289 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.497238    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw" podStartSLOduration=2.497224932 podStartE2EDuration="2.497224932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:13.49152804 +0000 UTC m=+1855.207873118" watchObservedRunningTime="2026-10-30 19:07:13.497224932 +0000 UTC m=+1855.213570008"
Oct 30 19:07:13.512937 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.512897    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv" podStartSLOduration=2.512885041 podStartE2EDuration="2.512885041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:13.508336394 +0000 UTC m=+1855.224681469" watchObservedRunningTime="2026-10-30 19:07:13.512885041 +0000 UTC m=+1855.229230160"
Oct 30 19:07:13.720695 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.720658    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"]
Oct 30 19:07:13.721253 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.721220    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" containerID="cri-o://fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32" gracePeriod=30
Oct 30 19:07:13.733245 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.732458    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"]
Oct 30 19:07:13.733245 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:13.732686    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy" containerID="cri-o://93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be" gracePeriod=30
Oct 30 19:07:14.050227 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:14.049486    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:14.683847 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:14.677675    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:15.373118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:15.349193    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:16.137022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:16.136989    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:16.413603 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:16.413569    4925 patch_prober.go:29] interesting pod/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:07:16.413844 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:16.413626    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:07:16.795239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:16.795145    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:17.140209 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:17.140122    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:17.711116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:17.704634    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:18.484127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:18.483643    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:19.065323 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.065292    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:07:19.138389 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.138363    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:07:19.227153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227071    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-kube-api-access-dpb6t\" (UniqueName: \"kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-kube-api-access-dpb6t\") pod \"52c08b4a-e175-4674-abc4-b783e98090d2\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") "
Oct 30 19:07:19.227342 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227166    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-certs\") pod \"33f803d9-d817-4b7a-97df-24609789f8e6\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") "
Oct 30 19:07:19.227342 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227297    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-credential-socket\") pod \"33f803d9-d817-4b7a-97df-24609789f8e6\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") "
Oct 30 19:07:19.227534 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227395    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-certs\") pod \"52c08b4a-e175-4674-abc4-b783e98090d2\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") "
Oct 30 19:07:19.227534 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227415    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-certs" pod "33f803d9-d817-4b7a-97df-24609789f8e6" (UID: "33f803d9-d817-4b7a-97df-24609789f8e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:19.227646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227592    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-kube-api-access-ms97t\" (UniqueName: \"kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-kube-api-access-ms97t\") pod \"33f803d9-d817-4b7a-97df-24609789f8e6\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") "
Oct 30 19:07:19.227707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227642    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-credential-socket" pod "33f803d9-d817-4b7a-97df-24609789f8e6" (UID: "33f803d9-d817-4b7a-97df-24609789f8e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:19.227707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227666    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-istio-token\" (UniqueName: \"kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-istio-token\") pod \"52c08b4a-e175-4674-abc4-b783e98090d2\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") "
Oct 30 19:07:19.227822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227711    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-envoy\") pod \"33f803d9-d817-4b7a-97df-24609789f8e6\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") "
Oct 30 19:07:19.227822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227744    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/33f803d9-d817-4b7a-97df-24609789f8e6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33f803d9-d817-4b7a-97df-24609789f8e6-istio-podinfo\") pod \"33f803d9-d817-4b7a-97df-24609789f8e6\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") "
Oct 30 19:07:19.227822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227796    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52c08b4a-e175-4674-abc4-b783e98090d2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c08b4a-e175-4674-abc4-b783e98090d2-istiod-ca-cert\") pod \"52c08b4a-e175-4674-abc4-b783e98090d2\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") "
Oct 30 19:07:19.227822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227818    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-certs" pod "52c08b4a-e175-4674-abc4-b783e98090d2" (UID: "52c08b4a-e175-4674-abc4-b783e98090d2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:19.228049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227827    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33f803d9-d817-4b7a-97df-24609789f8e6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33f803d9-d817-4b7a-97df-24609789f8e6-istiod-ca-cert\") pod \"33f803d9-d817-4b7a-97df-24609789f8e6\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") "
Oct 30 19:07:19.228049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227865    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-credential-socket\") pod \"52c08b4a-e175-4674-abc4-b783e98090d2\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") "
Oct 30 19:07:19.228049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227906    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-data\") pod \"52c08b4a-e175-4674-abc4-b783e98090d2\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") "
Oct 30 19:07:19.228049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.227992    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-istio-token\" (UniqueName: \"kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-istio-token\") pod \"33f803d9-d817-4b7a-97df-24609789f8e6\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") "
Oct 30 19:07:19.228049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.228026    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-envoy\") pod \"52c08b4a-e175-4674-abc4-b783e98090d2\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") "
Oct 30 19:07:19.228392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.228071    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-data\") pod \"33f803d9-d817-4b7a-97df-24609789f8e6\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") "
Oct 30 19:07:19.228392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.228295    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-socket\") pod \"33f803d9-d817-4b7a-97df-24609789f8e6\" (UID: \"33f803d9-d817-4b7a-97df-24609789f8e6\") "
Oct 30 19:07:19.228392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.228333    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/52c08b4a-e175-4674-abc4-b783e98090d2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c08b4a-e175-4674-abc4-b783e98090d2-istio-podinfo\") pod \"52c08b4a-e175-4674-abc4-b783e98090d2\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") "
Oct 30 19:07:19.228392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.228366    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-socket\") pod \"52c08b4a-e175-4674-abc4-b783e98090d2\" (UID: \"52c08b4a-e175-4674-abc4-b783e98090d2\") "
Oct 30 19:07:19.228640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.228617    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-credential-socket" pod "52c08b4a-e175-4674-abc4-b783e98090d2" (UID: "52c08b4a-e175-4674-abc4-b783e98090d2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:19.228846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.228819    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-data" pod "52c08b4a-e175-4674-abc4-b783e98090d2" (UID: "52c08b4a-e175-4674-abc4-b783e98090d2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:19.228985 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.228963    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-data" pod "33f803d9-d817-4b7a-97df-24609789f8e6" (UID: "33f803d9-d817-4b7a-97df-24609789f8e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:19.230090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.229188    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-socket" pod "33f803d9-d817-4b7a-97df-24609789f8e6" (UID: "33f803d9-d817-4b7a-97df-24609789f8e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:19.230090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.229359    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.230090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.229379    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.230090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.229393    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.230090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.229408    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.230090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.229422    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.230090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.229437    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.230090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.229449    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.230090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.229390    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-socket" pod "52c08b4a-e175-4674-abc4-b783e98090d2" (UID: "52c08b4a-e175-4674-abc4-b783e98090d2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:19.230090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.229953    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52c08b4a-e175-4674-abc4-b783e98090d2-istiod-ca-cert" pod "52c08b4a-e175-4674-abc4-b783e98090d2" (UID: "52c08b4a-e175-4674-abc4-b783e98090d2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:19.230090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.230057    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33f803d9-d817-4b7a-97df-24609789f8e6-istiod-ca-cert" pod "33f803d9-d817-4b7a-97df-24609789f8e6" (UID: "33f803d9-d817-4b7a-97df-24609789f8e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:19.232213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.231991    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-istio-token" pod "52c08b4a-e175-4674-abc4-b783e98090d2" (UID: "52c08b4a-e175-4674-abc4-b783e98090d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:19.232213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.232136    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-envoy" pod "52c08b4a-e175-4674-abc4-b783e98090d2" (UID: "52c08b4a-e175-4674-abc4-b783e98090d2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:19.232213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.232149    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-kube-api-access-ms97t" pod "33f803d9-d817-4b7a-97df-24609789f8e6" (UID: "33f803d9-d817-4b7a-97df-24609789f8e6"). InnerVolumeSpecName "kube-api-access-ms97t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:19.232892 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.232857    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-kube-api-access-dpb6t" pod "52c08b4a-e175-4674-abc4-b783e98090d2" (UID: "52c08b4a-e175-4674-abc4-b783e98090d2"). InnerVolumeSpecName "kube-api-access-dpb6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:19.233356 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.233332    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-envoy" pod "33f803d9-d817-4b7a-97df-24609789f8e6" (UID: "33f803d9-d817-4b7a-97df-24609789f8e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:19.233588 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.233566    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33f803d9-d817-4b7a-97df-24609789f8e6-istio-podinfo" pod "33f803d9-d817-4b7a-97df-24609789f8e6" (UID: "33f803d9-d817-4b7a-97df-24609789f8e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:07:19.233799 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.233778    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52c08b4a-e175-4674-abc4-b783e98090d2-istio-podinfo" pod "52c08b4a-e175-4674-abc4-b783e98090d2" (UID: "52c08b4a-e175-4674-abc4-b783e98090d2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:07:19.234255 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.234232    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-istio-token" pod "33f803d9-d817-4b7a-97df-24609789f8e6" (UID: "33f803d9-d817-4b7a-97df-24609789f8e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:19.317536 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.317164    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:19.347168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.339926    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c08b4a-e175-4674-abc4-b783e98090d2-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.347168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.339965    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.347168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.339980    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpb6t\" (UniqueName: \"kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-kube-api-access-dpb6t\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.347168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.339996    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms97t\" (UniqueName: \"kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-kube-api-access-ms97t\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.347168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.340010    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c08b4a-e175-4674-abc4-b783e98090d2-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.347168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.340024    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33f803d9-d817-4b7a-97df-24609789f8e6-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.347168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.340035    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33f803d9-d817-4b7a-97df-24609789f8e6-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.347168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.340048    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c08b4a-e175-4674-abc4-b783e98090d2-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.347168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.340063    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33f803d9-d817-4b7a-97df-24609789f8e6-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.347168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.340076    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33f803d9-d817-4b7a-97df-24609789f8e6-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.347168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.340089    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c08b4a-e175-4674-abc4-b783e98090d2-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:19.538413 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.538379    4925 generic.go:356] "Generic (PLEG): container finished" podID="52c08b4a-e175-4674-abc4-b783e98090d2" containerID="93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be" exitCode=0
Oct 30 19:07:19.538887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.538426    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"
Oct 30 19:07:19.538887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.538461    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" event={"ID":"52c08b4a-e175-4674-abc4-b783e98090d2","Type":"ContainerDied","Data":"93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be"}
Oct 30 19:07:19.538887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.538513    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz" event={"ID":"52c08b4a-e175-4674-abc4-b783e98090d2","Type":"ContainerDied","Data":"e48be01d2c1fcf49b838245859c7341217a54fca313bb520c87c84429999cabc"}
Oct 30 19:07:19.538887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.538537    4925 scope.go:122] "RemoveContainer" containerID="93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be"
Oct 30 19:07:19.540142 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.540118    4925 generic.go:356] "Generic (PLEG): container finished" podID="33f803d9-d817-4b7a-97df-24609789f8e6" containerID="fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32" exitCode=0
Oct 30 19:07:19.540448 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.540162    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" event={"ID":"33f803d9-d817-4b7a-97df-24609789f8e6","Type":"ContainerDied","Data":"fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32"}
Oct 30 19:07:19.540448 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.540201    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z" event={"ID":"33f803d9-d817-4b7a-97df-24609789f8e6","Type":"ContainerDied","Data":"ca9cdbcf639661b0e4ac0ef1ce093a3bd435886dd0ad484ffb5cd7306dfc84eb"}
Oct 30 19:07:19.540448 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.540204    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"
Oct 30 19:07:19.550956 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.550937    4925 scope.go:122] "RemoveContainer" containerID="93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be"
Oct 30 19:07:19.551257 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:19.551230    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be\": container with ID starting with 93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be not found: ID does not exist" containerID="93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be"
Oct 30 19:07:19.551357 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.551268    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be"} err="failed to get container status \"93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be\": rpc error: code = NotFound desc = could not find container \"93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be\": container with ID starting with 93895dd59726de0070cf651cba9da070d2a4aecd53bf8586a95f952c2e19a5be not found: ID does not exist"
Oct 30 19:07:19.551357 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.551290    4925 scope.go:122] "RemoveContainer" containerID="fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32"
Oct 30 19:07:19.561794 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.561775    4925 scope.go:122] "RemoveContainer" containerID="fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32"
Oct 30 19:07:19.562062 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:19.562043    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32\": container with ID starting with fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32 not found: ID does not exist" containerID="fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32"
Oct 30 19:07:19.562165 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.562067    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32"} err="failed to get container status \"fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32\": rpc error: code = NotFound desc = could not find container \"fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32\": container with ID starting with fa9b86c298d8f2b3ae527f710850da2c15e916dd8c6a15d68974d0a6829baf32 not found: ID does not exist"
Oct 30 19:07:19.576180 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.573201    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"]
Oct 30 19:07:19.581292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.581270    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-v6lzz"]
Oct 30 19:07:19.591682 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.591660    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"]
Oct 30 19:07:19.595768 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.595747    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-ltn6z"]
Oct 30 19:07:19.990127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:19.987516    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:20.425131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:20.425087    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" path="/var/lib/kubelet/pods/33f803d9-d817-4b7a-97df-24609789f8e6/volumes"
Oct 30 19:07:20.425642 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:20.425628    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" path="/var/lib/kubelet/pods/52c08b4a-e175-4674-abc4-b783e98090d2/volumes"
Oct 30 19:07:20.664218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:20.664175    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:21.539144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:21.535869    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:21.741122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:21.736188    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:22.460219 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:22.460191    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"
Oct 30 19:07:22.460902 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:22.460873    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"
Oct 30 19:07:22.977394 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:22.977361    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:23.228641 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:23.228547    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:23.494802 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:23.494701    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:23.703727 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:23.697142    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:24.035287 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.035253    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"]
Oct 30 19:07:24.036674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.036648    4925 cpu_manager.go:414] "containerMap: removing container" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy"
Oct 30 19:07:24.036674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.036674    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy"
Oct 30 19:07:24.036875 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.036702    4925 cpu_manager.go:414] "containerMap: removing container" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy"
Oct 30 19:07:24.036875 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.036711    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy"
Oct 30 19:07:24.036982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.036905    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="52c08b4a-e175-4674-abc4-b783e98090d2" containerName="istio-proxy"
Oct 30 19:07:24.036982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.036923    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="33f803d9-d817-4b7a-97df-24609789f8e6" containerName="istio-proxy"
Oct 30 19:07:24.043003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.042981    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.054523 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.054199    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-1dd09d-serving-cert\""
Oct 30 19:07:24.057166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.056337    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"]
Oct 30 19:07:24.194163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.193853    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:24.227161 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.227042    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50dfb53c-b7bf-43eb-8956-10836baafa25-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.227401 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.227362    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-metrics-certs\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.227501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.227462    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-maas-api-tls\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.227563 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.227498    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48nrg\" (UniqueName: \"kubernetes.io/projected/50dfb53c-b7bf-43eb-8956-10836baafa25-kube-api-access-48nrg\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.361167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.357559    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-metrics-certs\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.361167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.357680    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-maas-api-tls\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.361167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.357719    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48nrg\" (UniqueName: \"kubernetes.io/projected/50dfb53c-b7bf-43eb-8956-10836baafa25-kube-api-access-48nrg\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.361167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.357790    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50dfb53c-b7bf-43eb-8956-10836baafa25-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.361167 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:24.358229    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-serving-cert: secret "maas-api-e2e-extmod-a-1dd09d-serving-cert" not found
Oct 30 19:07:24.361167 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:24.358316    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-maas-api-tls podName:50dfb53c-b7bf-43eb-8956-10836baafa25 nodeName:}" failed. No retries permitted until 2026-10-30 19:07:24.858295561 +0000 UTC m=+1866.574640622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-maas-api-tls") pod "maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg" (UID: "50dfb53c-b7bf-43eb-8956-10836baafa25") : secret "maas-api-e2e-extmod-a-1dd09d-serving-cert" not found
Oct 30 19:07:24.361167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.358837    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50dfb53c-b7bf-43eb-8956-10836baafa25-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.404106 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.390610    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:24.404106 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.391572    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48nrg\" (UniqueName: \"kubernetes.io/projected/50dfb53c-b7bf-43eb-8956-10836baafa25-kube-api-access-48nrg\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.417162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.414568    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-metrics-certs\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.528659 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.528625    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"
Oct 30 19:07:24.664813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.664777    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:24.878123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.874494    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-maas-api-tls\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.908834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.905169    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-maas-api-tls\") pod \"maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:24.968494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:24.968390    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:25.371123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:25.363066    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:25.563261 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:25.562866    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:25.571779 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:25.571081    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"
Oct 30 19:07:25.778205 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:25.775399    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"]
Oct 30 19:07:26.767143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:26.766861    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:26.923838 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:26.923801    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg" event={"ID":"50dfb53c-b7bf-43eb-8956-10836baafa25","Type":"ContainerStarted","Data":"e24dad0f0607e1ba5266f197af29c602995e7368dff97db72fbf0693138cee3d"}
Oct 30 19:07:26.924038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:26.923871    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:26.924038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:26.923895    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg" event={"ID":"50dfb53c-b7bf-43eb-8956-10836baafa25","Type":"ContainerStarted","Data":"df5171fb49c9e65aef7a4d283b16f6f925d253a98f70d3c6ec31f497539e9262"}
Oct 30 19:07:26.950313 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:26.950266    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg" podStartSLOduration=2.46367838 podStartE2EDuration="2.950252478s" totalImagesPullingTime="486.574098ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:26.948802675 +0000 UTC m=+1868.665147756" watchObservedRunningTime="2026-10-30 19:07:26.950252478 +0000 UTC m=+1868.666597554"
Oct 30 19:07:27.104062 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:27.103775    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:27.319121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:27.316678    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:27.516141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:27.516077    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:27.697018 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:27.696980    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:28.032619 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.032554    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:28.254972 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.254942    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"]
Oct 30 19:07:28.259998 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.259978    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.264170 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.264118    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-f3d6a5-serving-cert\""
Oct 30 19:07:28.271440 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.271417    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"]
Oct 30 19:07:28.348476 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.348400    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.348657 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.348559    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-metrics-certs\") pod \"maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.348802 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.348782    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt259\" (UniqueName: \"kubernetes.io/projected/610e800d-7647-40fd-973c-bc58a0214728-kube-api-access-qt259\") pod \"maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.348941 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.348919    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/610e800d-7647-40fd-973c-bc58a0214728-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.446119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.442466    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:28.451531 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.451476    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.455158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.451891    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-metrics-certs\") pod \"maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.455717 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.455586    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt259\" (UniqueName: \"kubernetes.io/projected/610e800d-7647-40fd-973c-bc58a0214728-kube-api-access-qt259\") pod \"maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.456029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.455997    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/610e800d-7647-40fd-973c-bc58a0214728-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.457290 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.457269    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/610e800d-7647-40fd-973c-bc58a0214728-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.502125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.499738    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.529401 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.523634    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt259\" (UniqueName: \"kubernetes.io/projected/610e800d-7647-40fd-973c-bc58a0214728-kube-api-access-qt259\") pod \"maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.529401 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.523690    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-metrics-certs\") pod \"maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.578136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.577084    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:28.619815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.619748    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:28.761757 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.761721    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"]
Oct 30 19:07:28.782776 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.782228    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"
Oct 30 19:07:28.788687 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.788659    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-e51c10\""
Oct 30 19:07:28.803471 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.803328    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"]
Oct 30 19:07:28.816911 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.816884    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:28.817569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.817541    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"]
Oct 30 19:07:28.824274 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.824246    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"
Oct 30 19:07:28.825537 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.825511    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"]
Oct 30 19:07:28.829640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.829613    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-e51c10-dockercfg-tmltr\""
Oct 30 19:07:28.872177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.872117    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"]
Oct 30 19:07:28.872308 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:07:28.872271    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610e800d_7647_40fd_973c_bc58a0214728.slice/crio-32f2e3634084c68ff1b5668d1be03fe5ac58e916a6c31ef91bf47c5001044ada WatchSource:0}: Error finding container 32f2e3634084c68ff1b5668d1be03fe5ac58e916a6c31ef91bf47c5001044ada: Status 404 returned error can't find the container with id 32f2e3634084c68ff1b5668d1be03fe5ac58e916a6c31ef91bf47c5001044ada
Oct 30 19:07:28.886873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.886833    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvbrz\" (UniqueName: \"kubernetes.io/projected/1aeb450f-292b-4274-8bf4-2e8b6e686d50-kube-api-access-dvbrz\") pod \"payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf\" (UID: \"1aeb450f-292b-4274-8bf4-2e8b6e686d50\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"
Oct 30 19:07:28.886991 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.886899    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzhg7\" (UniqueName: \"kubernetes.io/projected/822c68de-9860-4a2d-b7ae-145bd319f117-kube-api-access-vzhg7\") pod \"payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2\" (UID: \"822c68de-9860-4a2d-b7ae-145bd319f117\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"
Oct 30 19:07:28.887066 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.886986    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1aeb450f-292b-4274-8bf4-2e8b6e686d50-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf\" (UID: \"1aeb450f-292b-4274-8bf4-2e8b6e686d50\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"
Oct 30 19:07:28.887254 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.887233    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/822c68de-9860-4a2d-b7ae-145bd319f117-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2\" (UID: \"822c68de-9860-4a2d-b7ae-145bd319f117\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"
Oct 30 19:07:28.946285 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.946251    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g" event={"ID":"610e800d-7647-40fd-973c-bc58a0214728","Type":"ContainerStarted","Data":"32f2e3634084c68ff1b5668d1be03fe5ac58e916a6c31ef91bf47c5001044ada"}
Oct 30 19:07:28.989255 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.989217    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvbrz\" (UniqueName: \"kubernetes.io/projected/1aeb450f-292b-4274-8bf4-2e8b6e686d50-kube-api-access-dvbrz\") pod \"payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf\" (UID: \"1aeb450f-292b-4274-8bf4-2e8b6e686d50\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"
Oct 30 19:07:28.989417 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.989276    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzhg7\" (UniqueName: \"kubernetes.io/projected/822c68de-9860-4a2d-b7ae-145bd319f117-kube-api-access-vzhg7\") pod \"payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2\" (UID: \"822c68de-9860-4a2d-b7ae-145bd319f117\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"
Oct 30 19:07:28.989417 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.989339    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1aeb450f-292b-4274-8bf4-2e8b6e686d50-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf\" (UID: \"1aeb450f-292b-4274-8bf4-2e8b6e686d50\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"
Oct 30 19:07:28.989536 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.989520    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/822c68de-9860-4a2d-b7ae-145bd319f117-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2\" (UID: \"822c68de-9860-4a2d-b7ae-145bd319f117\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"
Oct 30 19:07:28.990138 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.990115    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1aeb450f-292b-4274-8bf4-2e8b6e686d50-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf\" (UID: \"1aeb450f-292b-4274-8bf4-2e8b6e686d50\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"
Oct 30 19:07:28.990442 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.990419    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/822c68de-9860-4a2d-b7ae-145bd319f117-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2\" (UID: \"822c68de-9860-4a2d-b7ae-145bd319f117\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"
Oct 30 19:07:28.995326 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.994947    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:28.998792 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.998759    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzhg7\" (UniqueName: \"kubernetes.io/projected/822c68de-9860-4a2d-b7ae-145bd319f117-kube-api-access-vzhg7\") pod \"payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2\" (UID: \"822c68de-9860-4a2d-b7ae-145bd319f117\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"
Oct 30 19:07:28.999674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:28.999653    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvbrz\" (UniqueName: \"kubernetes.io/projected/1aeb450f-292b-4274-8bf4-2e8b6e686d50-kube-api-access-dvbrz\") pod \"payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf\" (UID: \"1aeb450f-292b-4274-8bf4-2e8b6e686d50\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"
Oct 30 19:07:29.140173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:29.129610    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"
Oct 30 19:07:29.156129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:29.150705    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"
Oct 30 19:07:29.850231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:29.833821    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:29.862926 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:29.835265    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"]
Oct 30 19:07:29.887967 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:07:29.887939    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aeb450f_292b_4274_8bf4_2e8b6e686d50.slice/crio-de5b87ffe295afb944bd5aa2b7bdef393f181324b84f445d747dde1ef98c8fd6 WatchSource:0}: Error finding container de5b87ffe295afb944bd5aa2b7bdef393f181324b84f445d747dde1ef98c8fd6: Status 404 returned error can't find the container with id de5b87ffe295afb944bd5aa2b7bdef393f181324b84f445d747dde1ef98c8fd6
Oct 30 19:07:29.900641 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:29.900441    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"]
Oct 30 19:07:29.903189 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:07:29.903162    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod822c68de_9860_4a2d_b7ae_145bd319f117.slice/crio-2ac2dfb825fa76386b2074e4edc1ee492eaf880f4b87e7581fd470459131f57b WatchSource:0}: Error finding container 2ac2dfb825fa76386b2074e4edc1ee492eaf880f4b87e7581fd470459131f57b: Status 404 returned error can't find the container with id 2ac2dfb825fa76386b2074e4edc1ee492eaf880f4b87e7581fd470459131f57b
Oct 30 19:07:29.957620 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:29.957591    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g" event={"ID":"610e800d-7647-40fd-973c-bc58a0214728","Type":"ContainerStarted","Data":"3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718"}
Oct 30 19:07:29.958365 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:29.958344    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:29.960458 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:29.960430    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2" event={"ID":"822c68de-9860-4a2d-b7ae-145bd319f117","Type":"ContainerStarted","Data":"2ac2dfb825fa76386b2074e4edc1ee492eaf880f4b87e7581fd470459131f57b"}
Oct 30 19:07:29.962449 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:29.962422    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf" event={"ID":"1aeb450f-292b-4274-8bf4-2e8b6e686d50","Type":"ContainerStarted","Data":"de5b87ffe295afb944bd5aa2b7bdef393f181324b84f445d747dde1ef98c8fd6"}
Oct 30 19:07:29.975952 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:29.975889    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g" podStartSLOduration=1.447338174 podStartE2EDuration="1.975874677s" totalImagesPullingTime="528.536503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:29.975226383 +0000 UTC m=+1871.691571459" watchObservedRunningTime="2026-10-30 19:07:29.975874677 +0000 UTC m=+1871.692219755"
Oct 30 19:07:30.187223 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.185555    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7493-f4531f"]
Oct 30 19:07:30.225459 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.225424    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7493-f4531f"
Oct 30 19:07:30.233670 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.226393    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7493-f4531f"]
Oct 30 19:07:30.237286 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.237242    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-c64f8\""
Oct 30 19:07:30.287063 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.287025    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p5sz\" (UniqueName: \"kubernetes.io/projected/8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1-kube-api-access-2p5sz\") pod \"test-curl-7493-f4531f\" (UID: \"8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1\") " pod="opendatahub/test-curl-7493-f4531f"
Oct 30 19:07:30.388847 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.388813    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p5sz\" (UniqueName: \"kubernetes.io/projected/8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1-kube-api-access-2p5sz\") pod \"test-curl-7493-f4531f\" (UID: \"8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1\") " pod="opendatahub/test-curl-7493-f4531f"
Oct 30 19:07:30.397472 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.397440    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p5sz\" (UniqueName: \"kubernetes.io/projected/8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1-kube-api-access-2p5sz\") pod \"test-curl-7493-f4531f\" (UID: \"8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1\") " pod="opendatahub/test-curl-7493-f4531f"
Oct 30 19:07:30.461976 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.461900    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"]
Oct 30 19:07:30.462149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.462125    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" containerName="istio-proxy" containerID="cri-o://de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14" gracePeriod=30
Oct 30 19:07:30.586400 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.586364    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7493-f4531f"
Oct 30 19:07:30.876507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.876478    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7493-f4531f"]
Oct 30 19:07:30.877597 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:07:30.877476    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a940b71_9bbd_4bc7_a23d_63bf6ddbcef1.slice/crio-7054625dd62ffc9002d78fbf127b5fe69b984e94e6cf4f94dc43519b5218cec3 WatchSource:0}: Error finding container 7054625dd62ffc9002d78fbf127b5fe69b984e94e6cf4f94dc43519b5218cec3: Status 404 returned error can't find the container with id 7054625dd62ffc9002d78fbf127b5fe69b984e94e6cf4f94dc43519b5218cec3
Oct 30 19:07:30.977427 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.977393    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf" event={"ID":"1aeb450f-292b-4274-8bf4-2e8b6e686d50","Type":"ContainerStarted","Data":"bae428d9a99cff9cc2a085b7b2ab681152e6d4c61d95b279f77e6b1dfa6f80cb"}
Oct 30 19:07:30.977546 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.977484    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"
Oct 30 19:07:30.979402 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.979337    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7493-f4531f" event={"ID":"8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1","Type":"ContainerStarted","Data":"a1bfe9ab8d118a376ba38ea9f4deb943c2c48c883c3e9ff0f0167da997aae2aa"}
Oct 30 19:07:30.979402 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.979380    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7493-f4531f" event={"ID":"8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1","Type":"ContainerStarted","Data":"7054625dd62ffc9002d78fbf127b5fe69b984e94e6cf4f94dc43519b5218cec3"}
Oct 30 19:07:30.981133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.981070    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2" event={"ID":"822c68de-9860-4a2d-b7ae-145bd319f117","Type":"ContainerStarted","Data":"8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8"}
Oct 30 19:07:30.981264 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:30.981249    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"
Oct 30 19:07:31.037314 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:31.035553    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf" podStartSLOduration=3.035530571 podStartE2EDuration="3.035530571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:31.008073654 +0000 UTC m=+1872.724418731" watchObservedRunningTime="2026-10-30 19:07:31.035530571 +0000 UTC m=+1872.751875647"
Oct 30 19:07:31.042419 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:31.039217    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:31.074452 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:31.074388    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2" podStartSLOduration=3.074364352 podStartE2EDuration="3.074364352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:31.028656448 +0000 UTC m=+1872.745001524" watchObservedRunningTime="2026-10-30 19:07:31.074364352 +0000 UTC m=+1872.790709429"
Oct 30 19:07:31.099696 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:31.099619    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7493-f4531f" podStartSLOduration=1.099597061 podStartE2EDuration="1.099597061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:31.048307798 +0000 UTC m=+1872.764652879" watchObservedRunningTime="2026-10-30 19:07:31.099597061 +0000 UTC m=+1872.815942139"
Oct 30 19:07:31.546236 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:31.546206    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:31.732033 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:31.731992    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:31.780394 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:31.780356    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7493-f4531f"]
Oct 30 19:07:31.787046 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:31.787014    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7493-f4531f"]
Oct 30 19:07:32.005800 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:32.005728    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:32.704784 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:32.704732    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:32.921138 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:32.910874    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:33.070182 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:33.066479    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:07:33.203495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:33.203457    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7493-f4531f" podUID="8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1" containerName="test-curl-7493-f4531f" containerID="cri-o://a1bfe9ab8d118a376ba38ea9f4deb943c2c48c883c3e9ff0f0167da997aae2aa" gracePeriod=2
Oct 30 19:07:33.755121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:33.745777    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:34.500128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:34.497380    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:35.247172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.241763    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:35.590593 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.590564    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7493-f4531f"
Oct 30 19:07:35.617467 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.617438    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1-kube-api-access-2p5sz\" (UniqueName: \"kubernetes.io/projected/8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1-kube-api-access-2p5sz\") pod \"8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1\" (UID: \"8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1\") "
Oct 30 19:07:35.621302 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.621268    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1-kube-api-access-2p5sz" pod "8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1" (UID: "8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1"). InnerVolumeSpecName "kube-api-access-2p5sz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:35.719130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.719040    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p5sz\" (UniqueName: \"kubernetes.io/projected/8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1-kube-api-access-2p5sz\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:35.767678 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.767651    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:07:35.832033 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.831988    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-credential-socket\") pod \"040bb776-e34b-4301-8bd0-634c316c46c5\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") "
Oct 30 19:07:35.832252 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.832076    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-data\") pod \"040bb776-e34b-4301-8bd0-634c316c46c5\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") "
Oct 30 19:07:35.832252 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.832217    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-envoy\") pod \"040bb776-e34b-4301-8bd0-634c316c46c5\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") "
Oct 30 19:07:35.832398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.832267    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-kube-api-access-hn5mj\" (UniqueName: \"kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-kube-api-access-hn5mj\") pod \"040bb776-e34b-4301-8bd0-634c316c46c5\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") "
Oct 30 19:07:35.832398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.832392    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-socket\") pod \"040bb776-e34b-4301-8bd0-634c316c46c5\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") "
Oct 30 19:07:35.832495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.832456    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/040bb776-e34b-4301-8bd0-634c316c46c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040bb776-e34b-4301-8bd0-634c316c46c5-istio-podinfo\") pod \"040bb776-e34b-4301-8bd0-634c316c46c5\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") "
Oct 30 19:07:35.832549 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.832510    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-istio-token\" (UniqueName: \"kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-istio-token\") pod \"040bb776-e34b-4301-8bd0-634c316c46c5\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") "
Oct 30 19:07:35.832638 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.832620    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/040bb776-e34b-4301-8bd0-634c316c46c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040bb776-e34b-4301-8bd0-634c316c46c5-istiod-ca-cert\") pod \"040bb776-e34b-4301-8bd0-634c316c46c5\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") "
Oct 30 19:07:35.832760 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.832744    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-certs\") pod \"040bb776-e34b-4301-8bd0-634c316c46c5\" (UID: \"040bb776-e34b-4301-8bd0-634c316c46c5\") "
Oct 30 19:07:35.834438 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.834197    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-certs" pod "040bb776-e34b-4301-8bd0-634c316c46c5" (UID: "040bb776-e34b-4301-8bd0-634c316c46c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:35.834826 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.834639    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-credential-socket" pod "040bb776-e34b-4301-8bd0-634c316c46c5" (UID: "040bb776-e34b-4301-8bd0-634c316c46c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:35.835267 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.834952    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-data" pod "040bb776-e34b-4301-8bd0-634c316c46c5" (UID: "040bb776-e34b-4301-8bd0-634c316c46c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:35.841970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.841944    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/040bb776-e34b-4301-8bd0-634c316c46c5-istiod-ca-cert" pod "040bb776-e34b-4301-8bd0-634c316c46c5" (UID: "040bb776-e34b-4301-8bd0-634c316c46c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:35.842256 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.842232    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-socket" pod "040bb776-e34b-4301-8bd0-634c316c46c5" (UID: "040bb776-e34b-4301-8bd0-634c316c46c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:35.842605 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.842557    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-kube-api-access-hn5mj" pod "040bb776-e34b-4301-8bd0-634c316c46c5" (UID: "040bb776-e34b-4301-8bd0-634c316c46c5"). InnerVolumeSpecName "kube-api-access-hn5mj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:35.863983 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.863897    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/040bb776-e34b-4301-8bd0-634c316c46c5-istio-podinfo" pod "040bb776-e34b-4301-8bd0-634c316c46c5" (UID: "040bb776-e34b-4301-8bd0-634c316c46c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:07:35.864137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.863984    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-envoy" pod "040bb776-e34b-4301-8bd0-634c316c46c5" (UID: "040bb776-e34b-4301-8bd0-634c316c46c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:35.865904 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.865874    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-istio-token" pod "040bb776-e34b-4301-8bd0-634c316c46c5" (UID: "040bb776-e34b-4301-8bd0-634c316c46c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:35.878061 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.878033    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:35.958947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.937984    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:35.958947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.938046    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/040bb776-e34b-4301-8bd0-634c316c46c5-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:35.958947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.938064    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:35.958947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.938078    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/040bb776-e34b-4301-8bd0-634c316c46c5-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:35.958947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.938113    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:35.958947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.938129    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:35.958947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.938142    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:35.958947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.938155    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/040bb776-e34b-4301-8bd0-634c316c46c5-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:35.958947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:35.938171    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn5mj\" (UniqueName: \"kubernetes.io/projected/040bb776-e34b-4301-8bd0-634c316c46c5-kube-api-access-hn5mj\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:36.011714 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.011617    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:07:36.268304 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.268207    4925 generic.go:356] "Generic (PLEG): container finished" podID="8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1" containerID="a1bfe9ab8d118a376ba38ea9f4deb943c2c48c883c3e9ff0f0167da997aae2aa" exitCode=137
Oct 30 19:07:36.268654 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.268460    4925 scope.go:122] "RemoveContainer" containerID="a1bfe9ab8d118a376ba38ea9f4deb943c2c48c883c3e9ff0f0167da997aae2aa"
Oct 30 19:07:36.268654 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.268649    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7493-f4531f"
Oct 30 19:07:36.271333 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.271310    4925 generic.go:356] "Generic (PLEG): container finished" podID="040bb776-e34b-4301-8bd0-634c316c46c5" containerID="de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14" exitCode=0
Oct 30 19:07:36.271475 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.271379    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"
Oct 30 19:07:36.271475 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.271412    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" event={"ID":"040bb776-e34b-4301-8bd0-634c316c46c5","Type":"ContainerDied","Data":"de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14"}
Oct 30 19:07:36.271475 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.271454    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn" event={"ID":"040bb776-e34b-4301-8bd0-634c316c46c5","Type":"ContainerDied","Data":"c1e51db9c14663ddc6404862190622521c45ec4cf789f0d9cdf504c1657c231a"}
Oct 30 19:07:36.284722 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.284625    4925 scope.go:122] "RemoveContainer" containerID="a1bfe9ab8d118a376ba38ea9f4deb943c2c48c883c3e9ff0f0167da997aae2aa"
Oct 30 19:07:36.285079 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:36.284930    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1bfe9ab8d118a376ba38ea9f4deb943c2c48c883c3e9ff0f0167da997aae2aa\": container with ID starting with a1bfe9ab8d118a376ba38ea9f4deb943c2c48c883c3e9ff0f0167da997aae2aa not found: ID does not exist" containerID="a1bfe9ab8d118a376ba38ea9f4deb943c2c48c883c3e9ff0f0167da997aae2aa"
Oct 30 19:07:36.285079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.284969    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1bfe9ab8d118a376ba38ea9f4deb943c2c48c883c3e9ff0f0167da997aae2aa"} err="failed to get container status \"a1bfe9ab8d118a376ba38ea9f4deb943c2c48c883c3e9ff0f0167da997aae2aa\": rpc error: code = NotFound desc = could not find container \"a1bfe9ab8d118a376ba38ea9f4deb943c2c48c883c3e9ff0f0167da997aae2aa\": container with ID starting with a1bfe9ab8d118a376ba38ea9f4deb943c2c48c883c3e9ff0f0167da997aae2aa not found: ID does not exist"
Oct 30 19:07:36.285079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.284993    4925 scope.go:122] "RemoveContainer" containerID="de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14"
Oct 30 19:07:36.288620 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:36.288593    4925 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7493-f4531f\" is forbidden: User \"system:node:ip-10-0-134-58.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-58.us-east-2.compute.internal' and this object" podUID="8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1" pod="opendatahub/test-curl-7493-f4531f"
Oct 30 19:07:36.299940 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.299882    4925 scope.go:122] "RemoveContainer" containerID="de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14"
Oct 30 19:07:36.300360 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:36.300333    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14\": container with ID starting with de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14 not found: ID does not exist" containerID="de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14"
Oct 30 19:07:36.300474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.300369    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14"} err="failed to get container status \"de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14\": rpc error: code = NotFound desc = could not find container \"de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14\": container with ID starting with de7dcc0b4ab90ac54b6a8d62e5ccdb095ed0f0f89c9ef72356fe1ba665b84b14 not found: ID does not exist"
Oct 30 19:07:36.307599 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:36.307570    4925 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7493-f4531f\" is forbidden: User \"system:node:ip-10-0-134-58.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-58.us-east-2.compute.internal' and this object" podUID="8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1" pod="opendatahub/test-curl-7493-f4531f"
Oct 30 19:07:36.308083 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.308063    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"]
Oct 30 19:07:36.313778 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.313756    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-wrkgn"]
Oct 30 19:07:36.426545 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.426511    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" path="/var/lib/kubelet/pods/040bb776-e34b-4301-8bd0-634c316c46c5/volumes"
Oct 30 19:07:36.427066 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.427050    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1" path="/var/lib/kubelet/pods/8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1/volumes"
Oct 30 19:07:36.688024 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:36.687982    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:37.475123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:37.470052    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:38.066113 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:38.066004    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:38.727358 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:38.727316    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:39.394764 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:39.394727    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:40.216168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:40.213821    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:40.870839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:40.870801    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:41.496701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:41.496668    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:42.017993 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:42.017965    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"
Oct 30 19:07:42.018209 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:42.018051    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"
Oct 30 19:07:42.105651 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:42.103757    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:43.061219 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.061175    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:43.304085 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:07:43.304049    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:07:43.751121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.743852    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"]
Oct 30 19:07:43.751121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.745299    4925 cpu_manager.go:414] "containerMap: removing container" podUID="8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1" containerName="test-curl-7493-f4531f"
Oct 30 19:07:43.751121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.745322    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1" containerName="test-curl-7493-f4531f"
Oct 30 19:07:43.751121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.745353    4925 cpu_manager.go:414] "containerMap: removing container" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" containerName="istio-proxy"
Oct 30 19:07:43.751121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.745360    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" containerName="istio-proxy"
Oct 30 19:07:43.751121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.745618    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="040bb776-e34b-4301-8bd0-634c316c46c5" containerName="istio-proxy"
Oct 30 19:07:43.751121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.745661    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a940b71-9bbd-4bc7-a23d-63bf6ddbcef1" containerName="test-curl-7493-f4531f"
Oct 30 19:07:43.793212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.791711    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:43.827193 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.807299    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-e51c10-serving-cert\""
Oct 30 19:07:43.827193 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.814426    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:43.828314 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.828226    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"]
Oct 30 19:07:43.862326 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.862287    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-metrics-certs\") pod \"maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:43.862518 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.862496    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jwtc\" (UniqueName: \"kubernetes.io/projected/5bb2e79e-a70d-48c3-b8f5-622717c110c3-kube-api-access-6jwtc\") pod \"maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:43.862711 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.862695    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-maas-api-tls\") pod \"maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:43.862786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.862749    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb2e79e-a70d-48c3-b8f5-622717c110c3-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:43.964557 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.964523    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-maas-api-tls\") pod \"maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:43.964728 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.964600    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb2e79e-a70d-48c3-b8f5-622717c110c3-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:43.964728 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.964690    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-metrics-certs\") pod \"maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:43.964864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.964847    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jwtc\" (UniqueName: \"kubernetes.io/projected/5bb2e79e-a70d-48c3-b8f5-622717c110c3-kube-api-access-6jwtc\") pod \"maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:43.965642 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.965612    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb2e79e-a70d-48c3-b8f5-622717c110c3-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:43.967422 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.967393    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-maas-api-tls\") pod \"maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:43.967821 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.967804    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-metrics-certs\") pod \"maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:43.973333 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:43.973309    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jwtc\" (UniqueName: \"kubernetes.io/projected/5bb2e79e-a70d-48c3-b8f5-622717c110c3-kube-api-access-6jwtc\") pod \"maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:44.035402 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:44.032466    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:44.132181 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:44.122619    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:44.214679 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:44.214653    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:44.595055 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:44.595024    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"]
Oct 30 19:07:44.826176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:44.824782    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:45.115749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:45.115703    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:45.354600 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:45.354500    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq" event={"ID":"5bb2e79e-a70d-48c3-b8f5-622717c110c3","Type":"ContainerStarted","Data":"1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58"}
Oct 30 19:07:45.355083 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:45.354611    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:45.355083 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:45.354639    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq" event={"ID":"5bb2e79e-a70d-48c3-b8f5-622717c110c3","Type":"ContainerStarted","Data":"ae46976ff78957a738053af221db9326ba7dd81a094323b33263e9241a6da814"}
Oct 30 19:07:45.381664 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:45.381605    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq" podStartSLOduration=1.917828605 podStartE2EDuration="2.381586754s" totalImagesPullingTime="463.758149ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:45.375040375 +0000 UTC m=+1887.091385470" watchObservedRunningTime="2026-10-30 19:07:45.381586754 +0000 UTC m=+1887.097931831"
Oct 30 19:07:45.594040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:45.594006    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:45.875618 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:45.875581    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:46.331452 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:46.331403    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:46.970124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:46.967918    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:47.934143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:47.933956    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:48.121474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:48.121438    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:48.321186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:48.309817    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:49.140481 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:49.140441    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:49.507847 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:49.507814    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:49.699556 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:49.699518    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:50.595830 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:50.595796    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"]
Oct 30 19:07:50.602839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:50.602814    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"
Oct 30 19:07:50.610302 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:50.610277    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-e67d95\""
Oct 30 19:07:50.615200 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:50.615169    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"]
Oct 30 19:07:50.654822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:50.654759    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8cks\" (UniqueName: \"kubernetes.io/projected/547ccaea-7848-46dd-a638-3411f31835f7-kube-api-access-t8cks\") pod \"payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd\" (UID: \"547ccaea-7848-46dd-a638-3411f31835f7\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"
Oct 30 19:07:50.654972 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:50.654856    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547ccaea-7848-46dd-a638-3411f31835f7-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd\" (UID: \"547ccaea-7848-46dd-a638-3411f31835f7\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"
Oct 30 19:07:50.782118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:50.764811    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8cks\" (UniqueName: \"kubernetes.io/projected/547ccaea-7848-46dd-a638-3411f31835f7-kube-api-access-t8cks\") pod \"payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd\" (UID: \"547ccaea-7848-46dd-a638-3411f31835f7\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"
Oct 30 19:07:50.782481 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:50.782451    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547ccaea-7848-46dd-a638-3411f31835f7-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd\" (UID: \"547ccaea-7848-46dd-a638-3411f31835f7\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"
Oct 30 19:07:50.807121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:50.783662    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547ccaea-7848-46dd-a638-3411f31835f7-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd\" (UID: \"547ccaea-7848-46dd-a638-3411f31835f7\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"
Oct 30 19:07:50.815356 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:50.812142    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8cks\" (UniqueName: \"kubernetes.io/projected/547ccaea-7848-46dd-a638-3411f31835f7-kube-api-access-t8cks\") pod \"payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd\" (UID: \"547ccaea-7848-46dd-a638-3411f31835f7\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"
Oct 30 19:07:50.934428 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:50.934389    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"
Oct 30 19:07:51.036621 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:51.036575    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:51.236141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:51.235193    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:51.372361 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:51.372329    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:07:51.457160 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:07:51.445235    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod547ccaea_7848_46dd_a638_3411f31835f7.slice/crio-d5e288d90c35f47a3445c2410ba9f011fcba918849a077c6f56b85f8074d9bda WatchSource:0}: Error finding container d5e288d90c35f47a3445c2410ba9f011fcba918849a077c6f56b85f8074d9bda: Status 404 returned error can't find the container with id d5e288d90c35f47a3445c2410ba9f011fcba918849a077c6f56b85f8074d9bda
Oct 30 19:07:51.457160 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:51.447142    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"]
Oct 30 19:07:51.676121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:51.672357    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:52.105148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:52.105114    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:52.312162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:52.305655    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:52.551720 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:52.548364    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:52.557598 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:52.557559    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd" event={"ID":"547ccaea-7848-46dd-a638-3411f31835f7","Type":"ContainerStarted","Data":"158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae"}
Oct 30 19:07:52.557776 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:52.557637    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"
Oct 30 19:07:52.557776 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:52.557665    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd" event={"ID":"547ccaea-7848-46dd-a638-3411f31835f7","Type":"ContainerStarted","Data":"d5e288d90c35f47a3445c2410ba9f011fcba918849a077c6f56b85f8074d9bda"}
Oct 30 19:07:52.577373 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:52.577326    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd" podStartSLOduration=2.577309633 podStartE2EDuration="2.577309633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:52.576638505 +0000 UTC m=+1894.292983581" watchObservedRunningTime="2026-10-30 19:07:52.577309633 +0000 UTC m=+1894.293654756"
Oct 30 19:07:52.754267 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:52.754231    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:53.024120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:53.023166    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:53.403479 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:53.403385    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:53.735626 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:53.735591    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:54.072819 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.072727    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"]
Oct 30 19:07:54.073421 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.072980    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" containerID="cri-o://3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c" gracePeriod=30
Oct 30 19:07:54.099977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.099946    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"]
Oct 30 19:07:54.110378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.109372    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.117956 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.117932    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f3d6a5\"/\"payload-processing-external-model-plugins-e2e-extmod-b-f3d6a5\""
Oct 30 19:07:54.118759 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.118733    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f3d6a5\"/\"routing-overlay\""
Oct 30 19:07:54.118898 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.118839    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f3d6a5\"/\"kube-root-ca.crt\""
Oct 30 19:07:54.118898 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.118895    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f3d6a5\"/\"openshift-service-ca.crt\""
Oct 30 19:07:54.123069 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.123043    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"]
Oct 30 19:07:54.153329 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.153274    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:54.188829 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.188796    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-f3d6a5\"/\"agc-cred-b58707117777\""
Oct 30 19:07:54.189013 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.188994    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-f3d6a5\"/\"payload-processing-external-model-e2e--eb883441-dockercfg-2ql8r\""
Oct 30 19:07:54.220122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.207776    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btqf2\" (UniqueName: \"kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-kube-api-access-btqf2\") pod \"payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") " pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.220122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.207886    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") " pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.220122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.208366    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") " pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.220122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.208416    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") " pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.318132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.311484    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") " pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.318132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.311552    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") " pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.318132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.311618    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btqf2\" (UniqueName: \"kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-kube-api-access-btqf2\") pod \"payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") " pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.318132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.311693    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") " pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.318132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.312782    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") " pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.358175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.344057    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") " pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.384118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.373677    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btqf2\" (UniqueName: \"kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-kube-api-access-btqf2\") pod \"payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") " pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.391153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.386645    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") " pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.402238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.400759    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:54.440366 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.439489    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:54.874122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:54.872993    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:55.080300 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:55.080270    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"]
Oct 30 19:07:55.081812 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:07:55.081782    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29a53a63_606a_4d0a_a548_b5f66cc2e988.slice/crio-e14643b01af2a5564dde5c47a6103bb895e91621c10dc260313840644788f52a WatchSource:0}: Error finding container e14643b01af2a5564dde5c47a6103bb895e91621c10dc260313840644788f52a: Status 404 returned error can't find the container with id e14643b01af2a5564dde5c47a6103bb895e91621c10dc260313840644788f52a
Oct 30 19:07:55.604129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:55.603695    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:55.911810 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:55.911775    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb" event={"ID":"29a53a63-606a-4d0a-a548-b5f66cc2e988","Type":"ContainerStarted","Data":"c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff"}
Oct 30 19:07:55.911974 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:55.911836    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb" event={"ID":"29a53a63-606a-4d0a-a548-b5f66cc2e988","Type":"ContainerStarted","Data":"e14643b01af2a5564dde5c47a6103bb895e91621c10dc260313840644788f52a"}
Oct 30 19:07:55.911974 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:55.911874    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:07:55.947469 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:55.947407    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb" podStartSLOduration=1.947389555 podStartE2EDuration="1.947389555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:55.940497865 +0000 UTC m=+1897.656842942" watchObservedRunningTime="2026-10-30 19:07:55.947389555 +0000 UTC m=+1897.663734632"
Oct 30 19:07:56.476872 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:56.476836    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:57.025292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:57.024720    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:57.406762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:57.405633    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:57.651048 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:57.651011    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:58.076188 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:58.076144    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:58.277213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:58.274310    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:58.839995 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:58.839958    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:07:58.840469 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:58.840018    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:07:58.913127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:58.912960    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:59.385445 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.385413    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:07:59.496487 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.496455    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0bf20214-a424-49a8-b45a-02236a44a19f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf20214-a424-49a8-b45a-02236a44a19f-istio-podinfo\") pod \"0bf20214-a424-49a8-b45a-02236a44a19f\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") "
Oct 30 19:07:59.496696 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.496582    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-kube-api-access-dbzdw\" (UniqueName: \"kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-kube-api-access-dbzdw\") pod \"0bf20214-a424-49a8-b45a-02236a44a19f\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") "
Oct 30 19:07:59.496696 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.496621    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-envoy\") pod \"0bf20214-a424-49a8-b45a-02236a44a19f\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") "
Oct 30 19:07:59.496696 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.496668    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-socket\") pod \"0bf20214-a424-49a8-b45a-02236a44a19f\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") "
Oct 30 19:07:59.496878 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.496846    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-istio-token\" (UniqueName: \"kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-istio-token\") pod \"0bf20214-a424-49a8-b45a-02236a44a19f\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") "
Oct 30 19:07:59.496937 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.496880    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0bf20214-a424-49a8-b45a-02236a44a19f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf20214-a424-49a8-b45a-02236a44a19f-istiod-ca-cert\") pod \"0bf20214-a424-49a8-b45a-02236a44a19f\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") "
Oct 30 19:07:59.496994 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.496947    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-credential-socket\") pod \"0bf20214-a424-49a8-b45a-02236a44a19f\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") "
Oct 30 19:07:59.497055 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.496994    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-data\") pod \"0bf20214-a424-49a8-b45a-02236a44a19f\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") "
Oct 30 19:07:59.497055 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.497027    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-certs\") pod \"0bf20214-a424-49a8-b45a-02236a44a19f\" (UID: \"0bf20214-a424-49a8-b45a-02236a44a19f\") "
Oct 30 19:07:59.497186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.497137    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-socket" pod "0bf20214-a424-49a8-b45a-02236a44a19f" (UID: "0bf20214-a424-49a8-b45a-02236a44a19f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:59.497754 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.497578    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-credential-socket" pod "0bf20214-a424-49a8-b45a-02236a44a19f" (UID: "0bf20214-a424-49a8-b45a-02236a44a19f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:59.497754 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.497647    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bf20214-a424-49a8-b45a-02236a44a19f-istiod-ca-cert" pod "0bf20214-a424-49a8-b45a-02236a44a19f" (UID: "0bf20214-a424-49a8-b45a-02236a44a19f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:59.498090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.498043    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-data" pod "0bf20214-a424-49a8-b45a-02236a44a19f" (UID: "0bf20214-a424-49a8-b45a-02236a44a19f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:59.498373 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.498342    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:59.498373 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.498365    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf20214-a424-49a8-b45a-02236a44a19f-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:59.498535 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.498381    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:59.498535 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.498396    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:59.498664 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.498529    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-certs" pod "0bf20214-a424-49a8-b45a-02236a44a19f" (UID: "0bf20214-a424-49a8-b45a-02236a44a19f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:59.499524 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.499479    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0bf20214-a424-49a8-b45a-02236a44a19f-istio-podinfo" pod "0bf20214-a424-49a8-b45a-02236a44a19f" (UID: "0bf20214-a424-49a8-b45a-02236a44a19f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:07:59.499524 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.499497    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-kube-api-access-dbzdw" pod "0bf20214-a424-49a8-b45a-02236a44a19f" (UID: "0bf20214-a424-49a8-b45a-02236a44a19f"). InnerVolumeSpecName "kube-api-access-dbzdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:59.499719 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.499693    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-istio-token" pod "0bf20214-a424-49a8-b45a-02236a44a19f" (UID: "0bf20214-a424-49a8-b45a-02236a44a19f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:59.500053 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.500033    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-envoy" pod "0bf20214-a424-49a8-b45a-02236a44a19f" (UID: "0bf20214-a424-49a8-b45a-02236a44a19f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:59.640057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.620969    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:59.640057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.621012    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf20214-a424-49a8-b45a-02236a44a19f-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:59.640057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.621030    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbzdw\" (UniqueName: \"kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-kube-api-access-dbzdw\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:59.640057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.621046    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf20214-a424-49a8-b45a-02236a44a19f-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:59.640057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.621061    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf20214-a424-49a8-b45a-02236a44a19f-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:59.877182 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.873697    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:59.920224 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.920184    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"]
Oct 30 19:07:59.920935 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.920893    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp" podUID="b36905cc-0aa9-4b0a-aca4-02ba5efae873" containerName="istio-proxy" containerID="cri-o://639a884f5ef7e03265f77b062348de14c4dc58f39eb6fe4c1335d7790ca01077" gracePeriod=30
Oct 30 19:07:59.936760 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.936723    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"]
Oct 30 19:07:59.936998 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:07:59.936970    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps" podUID="93bd9c54-8bda-4c70-962a-fe0f73e80a7a" containerName="istio-proxy" containerID="cri-o://695500f3785244e3e738f0029b8dda857ca6dd971ad313c18b28d23da2326abe" gracePeriod=30
Oct 30 19:08:00.018537 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:00.018504    4925 generic.go:356] "Generic (PLEG): container finished" podID="0bf20214-a424-49a8-b45a-02236a44a19f" containerID="3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c" exitCode=0
Oct 30 19:08:00.018726 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:00.018587    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"
Oct 30 19:08:00.018726 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:00.018586    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" event={"ID":"0bf20214-a424-49a8-b45a-02236a44a19f","Type":"ContainerDied","Data":"3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c"}
Oct 30 19:08:00.018726 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:00.018716    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws" event={"ID":"0bf20214-a424-49a8-b45a-02236a44a19f","Type":"ContainerDied","Data":"1f9dc71a75c6208c9b5fc5946f5958174998712a25573a4eb23675e6c0d9eb4c"}
Oct 30 19:08:00.018890 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:00.018743    4925 scope.go:122] "RemoveContainer" containerID="3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c"
Oct 30 19:08:00.032399 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:00.032380    4925 scope.go:122] "RemoveContainer" containerID="3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c"
Oct 30 19:08:00.032708 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:08:00.032687    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c\": container with ID starting with 3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c not found: ID does not exist" containerID="3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c"
Oct 30 19:08:00.032778 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:00.032721    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c"} err="failed to get container status \"3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c\": rpc error: code = NotFound desc = could not find container \"3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c\": container with ID starting with 3e261d544da8650c11e7212928585ef68256a80c644ea7683febd8b3979b8b2c not found: ID does not exist"
Oct 30 19:08:00.044696 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:00.044673    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"]
Oct 30 19:08:00.048835 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:00.048813    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-9kwws"]
Oct 30 19:08:00.425439 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:00.425409    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" path="/var/lib/kubelet/pods/0bf20214-a424-49a8-b45a-02236a44a19f/volumes"
Oct 30 19:08:01.090312 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:01.090208    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:02.071429 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:02.069494    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:02.288658 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:02.278601    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:02.528506 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:02.528444    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:03.265186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.265153    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"]
Oct 30 19:08:03.266536 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.266492    4925 cpu_manager.go:414] "containerMap: removing container" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy"
Oct 30 19:08:03.266536 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.266518    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy"
Oct 30 19:08:03.266765 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.266740    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="0bf20214-a424-49a8-b45a-02236a44a19f" containerName="istio-proxy"
Oct 30 19:08:03.272749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.272728    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"
Oct 30 19:08:03.284577 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.284553    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"]
Oct 30 19:08:03.284743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.284727    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-c164d6\""
Oct 30 19:08:03.374575 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.374539    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:03.450146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.449200    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/238cc3c0-8010-4e26-8eaa-927d1cdf823d-plugins-config-volume\") pod \"payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j\" (UID: \"238cc3c0-8010-4e26-8eaa-927d1cdf823d\") " pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"
Oct 30 19:08:03.450146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.449581    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfgkz\" (UniqueName: \"kubernetes.io/projected/238cc3c0-8010-4e26-8eaa-927d1cdf823d-kube-api-access-vfgkz\") pod \"payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j\" (UID: \"238cc3c0-8010-4e26-8eaa-927d1cdf823d\") " pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"
Oct 30 19:08:03.569230 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.562949    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/238cc3c0-8010-4e26-8eaa-927d1cdf823d-plugins-config-volume\") pod \"payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j\" (UID: \"238cc3c0-8010-4e26-8eaa-927d1cdf823d\") " pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"
Oct 30 19:08:03.569230 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.563434    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfgkz\" (UniqueName: \"kubernetes.io/projected/238cc3c0-8010-4e26-8eaa-927d1cdf823d-kube-api-access-vfgkz\") pod \"payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j\" (UID: \"238cc3c0-8010-4e26-8eaa-927d1cdf823d\") " pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"
Oct 30 19:08:03.585730 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.585695    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/238cc3c0-8010-4e26-8eaa-927d1cdf823d-plugins-config-volume\") pod \"payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j\" (UID: \"238cc3c0-8010-4e26-8eaa-927d1cdf823d\") " pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"
Oct 30 19:08:03.614022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.613987    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfgkz\" (UniqueName: \"kubernetes.io/projected/238cc3c0-8010-4e26-8eaa-927d1cdf823d-kube-api-access-vfgkz\") pod \"payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j\" (UID: \"238cc3c0-8010-4e26-8eaa-927d1cdf823d\") " pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"
Oct 30 19:08:03.617211 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.615312    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:03.757679 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.756285    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"
Oct 30 19:08:03.896833 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:03.896759    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"
Oct 30 19:08:04.083945 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:04.083910    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:04.205667 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:08:04.205640    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod238cc3c0_8010_4e26_8eaa_927d1cdf823d.slice/crio-efb4f28a9ea1f22c9659d2c44808d8627f1cf31b9d837d8ed1c9bf01f2bf6761 WatchSource:0}: Error finding container efb4f28a9ea1f22c9659d2c44808d8627f1cf31b9d837d8ed1c9bf01f2bf6761: Status 404 returned error can't find the container with id efb4f28a9ea1f22c9659d2c44808d8627f1cf31b9d837d8ed1c9bf01f2bf6761
Oct 30 19:08:04.207791 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:04.207754    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"]
Oct 30 19:08:04.504295 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:04.504205    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:05.142374 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.142338    4925 generic.go:356] "Generic (PLEG): container finished" podID="93bd9c54-8bda-4c70-962a-fe0f73e80a7a" containerID="695500f3785244e3e738f0029b8dda857ca6dd971ad313c18b28d23da2326abe" exitCode=0
Oct 30 19:08:05.142546 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.142402    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps" event={"ID":"93bd9c54-8bda-4c70-962a-fe0f73e80a7a","Type":"ContainerDied","Data":"695500f3785244e3e738f0029b8dda857ca6dd971ad313c18b28d23da2326abe"}
Oct 30 19:08:05.145490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.145466    4925 generic.go:356] "Generic (PLEG): container finished" podID="b36905cc-0aa9-4b0a-aca4-02ba5efae873" containerID="639a884f5ef7e03265f77b062348de14c4dc58f39eb6fe4c1335d7790ca01077" exitCode=0
Oct 30 19:08:05.145631 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.145540    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp" event={"ID":"b36905cc-0aa9-4b0a-aca4-02ba5efae873","Type":"ContainerDied","Data":"639a884f5ef7e03265f77b062348de14c4dc58f39eb6fe4c1335d7790ca01077"}
Oct 30 19:08:05.147621 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.147597    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j" event={"ID":"238cc3c0-8010-4e26-8eaa-927d1cdf823d","Type":"ContainerStarted","Data":"b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0"}
Oct 30 19:08:05.147713 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.147652    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j" event={"ID":"238cc3c0-8010-4e26-8eaa-927d1cdf823d","Type":"ContainerStarted","Data":"efb4f28a9ea1f22c9659d2c44808d8627f1cf31b9d837d8ed1c9bf01f2bf6761"}
Oct 30 19:08:05.147713 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.147686    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"
Oct 30 19:08:05.182163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.179156    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j" podStartSLOduration=2.179135071 podStartE2EDuration="2.179135071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:05.165471479 +0000 UTC m=+1906.881816557" watchObservedRunningTime="2026-10-30 19:08:05.179135071 +0000 UTC m=+1906.895480149"
Oct 30 19:08:05.317850 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.317826    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:08:05.420659 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.420558    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-envoy\") pod \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") "
Oct 30 19:08:05.425127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.425077    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istiod-ca-cert\") pod \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") "
Oct 30 19:08:05.426260 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.425392    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-certs\") pod \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") "
Oct 30 19:08:05.426524 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.426503    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-credential-socket\") pod \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") "
Oct 30 19:08:05.426652 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.426639    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-socket\") pod \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") "
Oct 30 19:08:05.426751 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.426740    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-token\" (UniqueName: \"kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-token\") pod \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") "
Oct 30 19:08:05.426948 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.426922    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-data\") pod \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") "
Oct 30 19:08:05.427046 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.427019    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-credential-socket" pod "b36905cc-0aa9-4b0a-aca4-02ba5efae873" (UID: "b36905cc-0aa9-4b0a-aca4-02ba5efae873"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:05.427951 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.427271    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-certs" pod "b36905cc-0aa9-4b0a-aca4-02ba5efae873" (UID: "b36905cc-0aa9-4b0a-aca4-02ba5efae873"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:05.427951 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.427463    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-socket" pod "b36905cc-0aa9-4b0a-aca4-02ba5efae873" (UID: "b36905cc-0aa9-4b0a-aca4-02ba5efae873"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:05.427951 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.427945    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istiod-ca-cert" pod "b36905cc-0aa9-4b0a-aca4-02ba5efae873" (UID: "b36905cc-0aa9-4b0a-aca4-02ba5efae873"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:05.429650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.428317    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-kube-api-access-m74vf\" (UniqueName: \"kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-kube-api-access-m74vf\") pod \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") "
Oct 30 19:08:05.429650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.428398    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-podinfo\") pod \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\" (UID: \"b36905cc-0aa9-4b0a-aca4-02ba5efae873\") "
Oct 30 19:08:05.429650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.428718    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-data" pod "b36905cc-0aa9-4b0a-aca4-02ba5efae873" (UID: "b36905cc-0aa9-4b0a-aca4-02ba5efae873"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:05.429650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.429284    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.429650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.429308    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.429650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.429324    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.429650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.429340    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.429650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.429354    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.430151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.429934    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:05.433193 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.431493    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:08:05.439040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.439013    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-kube-api-access-m74vf" pod "b36905cc-0aa9-4b0a-aca4-02ba5efae873" (UID: "b36905cc-0aa9-4b0a-aca4-02ba5efae873"). InnerVolumeSpecName "kube-api-access-m74vf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:05.441220 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.441193    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-podinfo" pod "b36905cc-0aa9-4b0a-aca4-02ba5efae873" (UID: "b36905cc-0aa9-4b0a-aca4-02ba5efae873"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:08:05.452777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.452735    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-envoy" pod "b36905cc-0aa9-4b0a-aca4-02ba5efae873" (UID: "b36905cc-0aa9-4b0a-aca4-02ba5efae873"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:05.461155 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.455211    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-token" pod "b36905cc-0aa9-4b0a-aca4-02ba5efae873" (UID: "b36905cc-0aa9-4b0a-aca4-02ba5efae873"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:05.530840 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.530760    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-certs\") pod \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") "
Oct 30 19:08:05.531324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.530824    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-podinfo\") pod \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") "
Oct 30 19:08:05.531324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.530894    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-kube-api-access-nvgb6\" (UniqueName: \"kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-kube-api-access-nvgb6\") pod \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") "
Oct 30 19:08:05.531324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.530937    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-credential-socket\") pod \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") "
Oct 30 19:08:05.531324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.531081    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-envoy\") pod \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") "
Oct 30 19:08:05.531324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.531146    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-socket\") pod \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") "
Oct 30 19:08:05.531324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.531201    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-data\") pod \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") "
Oct 30 19:08:05.531324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.531223    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istiod-ca-cert\") pod \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") "
Oct 30 19:08:05.531740 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.531417    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-token\" (UniqueName: \"kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-token\") pod \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\" (UID: \"93bd9c54-8bda-4c70-962a-fe0f73e80a7a\") "
Oct 30 19:08:05.531740 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.531572    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-certs" pod "93bd9c54-8bda-4c70-962a-fe0f73e80a7a" (UID: "93bd9c54-8bda-4c70-962a-fe0f73e80a7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:05.531866 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.531804    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-data" pod "93bd9c54-8bda-4c70-962a-fe0f73e80a7a" (UID: "93bd9c54-8bda-4c70-962a-fe0f73e80a7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:05.532640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.532218    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-socket" pod "93bd9c54-8bda-4c70-962a-fe0f73e80a7a" (UID: "93bd9c54-8bda-4c70-962a-fe0f73e80a7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:05.532640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.532430    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m74vf\" (UniqueName: \"kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-kube-api-access-m74vf\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.532640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.532460    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.532640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.532475    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.532640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.532491    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.532640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.532505    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.532640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.532518    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.532640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.532531    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b36905cc-0aa9-4b0a-aca4-02ba5efae873-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.532640 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.532434    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istiod-ca-cert" pod "93bd9c54-8bda-4c70-962a-fe0f73e80a7a" (UID: "93bd9c54-8bda-4c70-962a-fe0f73e80a7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:05.533412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.533301    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-credential-socket" pod "93bd9c54-8bda-4c70-962a-fe0f73e80a7a" (UID: "93bd9c54-8bda-4c70-962a-fe0f73e80a7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:05.534939 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.534917    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-kube-api-access-nvgb6" pod "93bd9c54-8bda-4c70-962a-fe0f73e80a7a" (UID: "93bd9c54-8bda-4c70-962a-fe0f73e80a7a"). InnerVolumeSpecName "kube-api-access-nvgb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:05.535415 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.535391    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-podinfo" pod "93bd9c54-8bda-4c70-962a-fe0f73e80a7a" (UID: "93bd9c54-8bda-4c70-962a-fe0f73e80a7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:08:05.535849 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.535829    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-token" pod "93bd9c54-8bda-4c70-962a-fe0f73e80a7a" (UID: "93bd9c54-8bda-4c70-962a-fe0f73e80a7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:05.536814 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.536796    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-envoy" pod "93bd9c54-8bda-4c70-962a-fe0f73e80a7a" (UID: "93bd9c54-8bda-4c70-962a-fe0f73e80a7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:05.633927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.633895    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.633927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.633926    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvgb6\" (UniqueName: \"kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-kube-api-access-nvgb6\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.633927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.633936    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.634195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.633946    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.634195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.633955    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:05.634195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:05.633963    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93bd9c54-8bda-4c70-962a-fe0f73e80a7a-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:06.184132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.183942    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:06.200810 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.199792    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps" event={"ID":"93bd9c54-8bda-4c70-962a-fe0f73e80a7a","Type":"ContainerDied","Data":"a98f9c3ff2ba8cf42d3fd572ac75b9d8dda7f4603a7d292119559568e6452434"}
Oct 30 19:08:06.200810 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.199863    4925 scope.go:122] "RemoveContainer" containerID="695500f3785244e3e738f0029b8dda857ca6dd971ad313c18b28d23da2326abe"
Oct 30 19:08:06.200810 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.200016    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"
Oct 30 19:08:06.232119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.231907    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"
Oct 30 19:08:06.232119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.231968    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp" event={"ID":"b36905cc-0aa9-4b0a-aca4-02ba5efae873","Type":"ContainerDied","Data":"a50fbc46176a101dbd815a4c5b6939eeb054aa911931147d0da685893fe2e0ff"}
Oct 30 19:08:06.242862 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.242838    4925 scope.go:122] "RemoveContainer" containerID="639a884f5ef7e03265f77b062348de14c4dc58f39eb6fe4c1335d7790ca01077"
Oct 30 19:08:06.253882 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.253855    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"]
Oct 30 19:08:06.258162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.258140    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-nbpps"]
Oct 30 19:08:06.269871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.269846    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"]
Oct 30 19:08:06.275993 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.275973    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-m49gp"]
Oct 30 19:08:06.432790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.432760    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93bd9c54-8bda-4c70-962a-fe0f73e80a7a" path="/var/lib/kubelet/pods/93bd9c54-8bda-4c70-962a-fe0f73e80a7a/volumes"
Oct 30 19:08:06.433282 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.433267    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b36905cc-0aa9-4b0a-aca4-02ba5efae873" path="/var/lib/kubelet/pods/b36905cc-0aa9-4b0a-aca4-02ba5efae873/volumes"
Oct 30 19:08:06.942124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:06.940333    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:08:07.363941 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:07.363852    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:07.974042 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:07.973999    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:08.172749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:08.172715    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:08.723723 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:08.723684    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:09.115716 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.115630    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"]
Oct 30 19:08:09.116293 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.115849    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" containerID="cri-o://f6ca2dbc120b4727fa87611a6d1cb758d76ad796ff65404298c2c83705da832d" gracePeriod=30
Oct 30 19:08:09.122134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.122087    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"]
Oct 30 19:08:09.122497 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.122325    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" containerID="cri-o://14dffd84df729d6f05a07c47f828692bfbcf3b7e62002646d744cc98c79678ed" gracePeriod=30
Oct 30 19:08:09.321508 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.321462    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:09.508657 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.508346    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:09.800390 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.800299    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"]
Oct 30 19:08:09.842272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.842232    4925 cpu_manager.go:414] "containerMap: removing container" podUID="93bd9c54-8bda-4c70-962a-fe0f73e80a7a" containerName="istio-proxy"
Oct 30 19:08:09.842272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.842268    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93bd9c54-8bda-4c70-962a-fe0f73e80a7a" containerName="istio-proxy"
Oct 30 19:08:09.842272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.842281    4925 cpu_manager.go:414] "containerMap: removing container" podUID="b36905cc-0aa9-4b0a-aca4-02ba5efae873" containerName="istio-proxy"
Oct 30 19:08:09.842587 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.842290    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b36905cc-0aa9-4b0a-aca4-02ba5efae873" containerName="istio-proxy"
Oct 30 19:08:09.842587 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.842530    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="93bd9c54-8bda-4c70-962a-fe0f73e80a7a" containerName="istio-proxy"
Oct 30 19:08:09.842587 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.842547    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="b36905cc-0aa9-4b0a-aca4-02ba5efae873" containerName="istio-proxy"
Oct 30 19:08:09.913117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.903438    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"]
Oct 30 19:08:09.913117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.903635    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:09.945137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.929533    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:09.946804 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.946603    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-28c1e9-openshift-default-dockercfg-g24g7\""
Oct 30 19:08:10.001140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.992709    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-envoy\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.001140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.992805    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-token\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.001140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.992851    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sndz7\" (UniqueName: \"kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-kube-api-access-sndz7\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.001140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.993013    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istiod-ca-cert\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.001140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.993081    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-socket\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.001140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.993128    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-data\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.001140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.993160    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-credential-socket\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.001140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.993313    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-certs\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.001140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:09.993390    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-podinfo\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.095225 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.094869    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-podinfo\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.095225 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.094965    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-envoy\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.095225 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.095045    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-token\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.095225 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.095088    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sndz7\" (UniqueName: \"kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-kube-api-access-sndz7\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.095570 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.095269    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istiod-ca-cert\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.095570 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.095336    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-socket\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.095570 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.095360    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-data\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.095570 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.095387    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-credential-socket\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.095570 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.095468    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-certs\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.095958 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.095915    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-certs\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.099342 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.097195    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istiod-ca-cert\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.099342 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.098131    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-data\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.099342 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.098431    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-socket\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.099342 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.098667    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-credential-socket\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.100577 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.100490    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-envoy\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.102274 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.102235    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-podinfo\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.107680 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.107654    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-token\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.109415 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.108896    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sndz7\" (UniqueName: \"kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-kube-api-access-sndz7\") pod \"e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") " pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.233902 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.233851    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:08:10.234421 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.233909    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:08:10.392170 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.391658    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:10.643205 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.642933    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"]
Oct 30 19:08:10.649501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.649468    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:10.656165 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.655813    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-e67d95-serving-cert\""
Oct 30 19:08:10.671171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.671144    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"]
Oct 30 19:08:10.718713 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.718550    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe7bcca5-db06-4385-8f07-a402030e64b1-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:10.718815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.718781    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-metrics-certs\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:10.718949 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.718925    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:10.719404 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.719364    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvt72\" (UniqueName: \"kubernetes.io/projected/fe7bcca5-db06-4385-8f07-a402030e64b1-kube-api-access-vvt72\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:10.794176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.794078    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:10.847132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.846932    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvt72\" (UniqueName: \"kubernetes.io/projected/fe7bcca5-db06-4385-8f07-a402030e64b1-kube-api-access-vvt72\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:10.847324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.847158    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe7bcca5-db06-4385-8f07-a402030e64b1-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:10.847440 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.847335    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-metrics-certs\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:10.872348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.847496    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:10.872348 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:08:10.847775    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-serving-cert: secret "maas-api-e2e-extmod-b-e67d95-serving-cert" not found
Oct 30 19:08:10.872348 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:08:10.847847    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls podName:fe7bcca5-db06-4385-8f07-a402030e64b1 nodeName:}" failed. No retries permitted until 2026-10-30 19:08:11.347824067 +0000 UTC m=+1913.064169135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls") pod "maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph" (UID: "fe7bcca5-db06-4385-8f07-a402030e64b1") : secret "maas-api-e2e-extmod-b-e67d95-serving-cert" not found
Oct 30 19:08:10.872348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.849262    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe7bcca5-db06-4385-8f07-a402030e64b1-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:10.887641 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.887599    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvt72\" (UniqueName: \"kubernetes.io/projected/fe7bcca5-db06-4385-8f07-a402030e64b1-kube-api-access-vvt72\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:10.924013 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.923926    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-metrics-certs\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:10.987872 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.987835    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"]
Oct 30 19:08:10.994130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.993398    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:10.997140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.995824    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:08:10.997140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.995913    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:08:10.997140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:10.995956    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:08:11.283089 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:11.283003    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" event={"ID":"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8","Type":"ContainerStarted","Data":"c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634"}
Oct 30 19:08:11.283089 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:11.283058    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" event={"ID":"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8","Type":"ContainerStarted","Data":"7da4a73f9d59e1029f4359024f7fcc11c8099a6caec8c17197c655df7c39a38d"}
Oct 30 19:08:11.314129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:11.309836    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" podStartSLOduration=2.309820434 podStartE2EDuration="2.309820434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:11.30549967 +0000 UTC m=+1913.021844748" watchObservedRunningTime="2026-10-30 19:08:11.309820434 +0000 UTC m=+1913.026165514"
Oct 30 19:08:11.376193 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:11.374190    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:11.376193 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:08:11.375040    4925 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-serving-cert: secret "maas-api-e2e-extmod-b-e67d95-serving-cert" not found
Oct 30 19:08:11.376193 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:08:11.375150    4925 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls podName:fe7bcca5-db06-4385-8f07-a402030e64b1 nodeName:}" failed. No retries permitted until 2026-10-30 19:08:12.375083075 +0000 UTC m=+1914.091428146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls") pod "maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph" (UID: "fe7bcca5-db06-4385-8f07-a402030e64b1") : secret "maas-api-e2e-extmod-b-e67d95-serving-cert" not found
Oct 30 19:08:11.404123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:11.394673    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:11.629177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:11.627903    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:11.931353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:11.931313    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:12.130892 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:12.130859    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:12.400870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:12.400781    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:12.409050 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:12.409023    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:12.411820 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:12.411801    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls\") pod \"maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:12.481417 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:12.481389    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:12.715295 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:12.713462    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:12.762666 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:12.762627    4925 patch_prober.go:29] interesting pod/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:08:12.762853 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:12.762698    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:08:12.803628 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:12.803592    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"]
Oct 30 19:08:12.807509 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:08:12.807479    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe7bcca5_db06_4385_8f07_a402030e64b1.slice/crio-e0aa0837a0334f0e30cd297241f1a793772e48f74966a50615e7af8faafab49e WatchSource:0}: Error finding container e0aa0837a0334f0e30cd297241f1a793772e48f74966a50615e7af8faafab49e: Status 404 returned error can't find the container with id e0aa0837a0334f0e30cd297241f1a793772e48f74966a50615e7af8faafab49e
Oct 30 19:08:13.302749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:13.302495    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph" event={"ID":"fe7bcca5-db06-4385-8f07-a402030e64b1","Type":"ContainerStarted","Data":"e0aa0837a0334f0e30cd297241f1a793772e48f74966a50615e7af8faafab49e"}
Oct 30 19:08:13.303286 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:13.303261    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:13.304570 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:13.304546    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:08:13.309348 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:08:13.309318    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:08:13.316087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:13.316056    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:13.521484 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:13.521385    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:13.763495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:13.762080    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:14.096423 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.095647    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:14.288635 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.283825    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:14.333436 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.333402    4925 generic.go:356] "Generic (PLEG): container finished" podID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerID="f6ca2dbc120b4727fa87611a6d1cb758d76ad796ff65404298c2c83705da832d" exitCode=0
Oct 30 19:08:14.333953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.333920    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" event={"ID":"86c005ba-9d2d-4d5d-aeac-2e762809092c","Type":"ContainerDied","Data":"f6ca2dbc120b4727fa87611a6d1cb758d76ad796ff65404298c2c83705da832d"}
Oct 30 19:08:14.336714 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.336687    4925 generic.go:356] "Generic (PLEG): container finished" podID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerID="14dffd84df729d6f05a07c47f828692bfbcf3b7e62002646d744cc98c79678ed" exitCode=0
Oct 30 19:08:14.336914 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.336894    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" event={"ID":"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73","Type":"ContainerDied","Data":"14dffd84df729d6f05a07c47f828692bfbcf3b7e62002646d744cc98c79678ed"}
Oct 30 19:08:14.343011 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.342955    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph" event={"ID":"fe7bcca5-db06-4385-8f07-a402030e64b1","Type":"ContainerStarted","Data":"fee78f70ec9dc1f86d4c4bf1087464fc527b64a8278f4ff7f0629be813bd8f8e"}
Oct 30 19:08:14.343011 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.343011    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:14.367758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.367667    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph" podStartSLOduration=3.927960515 podStartE2EDuration="4.367649376s" totalImagesPullingTime="439.688861ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:14.362414542 +0000 UTC m=+1916.078759619" watchObservedRunningTime="2026-10-30 19:08:14.367649376 +0000 UTC m=+1916.083994452"
Oct 30 19:08:14.479446 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.479419    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:08:14.575904 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.562651    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-kube-api-access-mbgcp\" (UniqueName: \"kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-kube-api-access-mbgcp\") pod \"86c005ba-9d2d-4d5d-aeac-2e762809092c\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") "
Oct 30 19:08:14.599625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.598746    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-credential-socket\") pod \"86c005ba-9d2d-4d5d-aeac-2e762809092c\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") "
Oct 30 19:08:14.599625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.598935    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-token\" (UniqueName: \"kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-token\") pod \"86c005ba-9d2d-4d5d-aeac-2e762809092c\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") "
Oct 30 19:08:14.599625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.599065    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-credential-socket" pod "86c005ba-9d2d-4d5d-aeac-2e762809092c" (UID: "86c005ba-9d2d-4d5d-aeac-2e762809092c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:14.599625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.599345    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-kube-api-access-mbgcp" pod "86c005ba-9d2d-4d5d-aeac-2e762809092c" (UID: "86c005ba-9d2d-4d5d-aeac-2e762809092c"). InnerVolumeSpecName "kube-api-access-mbgcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:14.599625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.599362    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-envoy\") pod \"86c005ba-9d2d-4d5d-aeac-2e762809092c\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") "
Oct 30 19:08:14.600026 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.599662    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-certs\") pod \"86c005ba-9d2d-4d5d-aeac-2e762809092c\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") "
Oct 30 19:08:14.600026 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.599716    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86c005ba-9d2d-4d5d-aeac-2e762809092c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c005ba-9d2d-4d5d-aeac-2e762809092c-istiod-ca-cert\") pod \"86c005ba-9d2d-4d5d-aeac-2e762809092c\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") "
Oct 30 19:08:14.600026 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.599873    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-data\") pod \"86c005ba-9d2d-4d5d-aeac-2e762809092c\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") "
Oct 30 19:08:14.600026 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.599948    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-podinfo\") pod \"86c005ba-9d2d-4d5d-aeac-2e762809092c\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") "
Oct 30 19:08:14.600250 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.600037    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-socket\") pod \"86c005ba-9d2d-4d5d-aeac-2e762809092c\" (UID: \"86c005ba-9d2d-4d5d-aeac-2e762809092c\") "
Oct 30 19:08:14.606690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.600958    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbgcp\" (UniqueName: \"kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-kube-api-access-mbgcp\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.606690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.600981    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.606690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.601655    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-certs" pod "86c005ba-9d2d-4d5d-aeac-2e762809092c" (UID: "86c005ba-9d2d-4d5d-aeac-2e762809092c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:14.606690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.601959    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86c005ba-9d2d-4d5d-aeac-2e762809092c-istiod-ca-cert" pod "86c005ba-9d2d-4d5d-aeac-2e762809092c" (UID: "86c005ba-9d2d-4d5d-aeac-2e762809092c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:14.606690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.602230    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-data" pod "86c005ba-9d2d-4d5d-aeac-2e762809092c" (UID: "86c005ba-9d2d-4d5d-aeac-2e762809092c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:14.606690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.602927    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-socket" pod "86c005ba-9d2d-4d5d-aeac-2e762809092c" (UID: "86c005ba-9d2d-4d5d-aeac-2e762809092c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:14.625618 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.625573    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-token" pod "86c005ba-9d2d-4d5d-aeac-2e762809092c" (UID: "86c005ba-9d2d-4d5d-aeac-2e762809092c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:14.628199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.628176    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-podinfo" pod "86c005ba-9d2d-4d5d-aeac-2e762809092c" (UID: "86c005ba-9d2d-4d5d-aeac-2e762809092c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:08:14.636683 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.636574    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-envoy" pod "86c005ba-9d2d-4d5d-aeac-2e762809092c" (UID: "86c005ba-9d2d-4d5d-aeac-2e762809092c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:14.673194 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.669548    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:14.737815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.733568    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.737815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.733609    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.737815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.733627    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.737815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.733640    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.737815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.733653    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.737815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.733666    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c005ba-9d2d-4d5d-aeac-2e762809092c-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.737815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.733680    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c005ba-9d2d-4d5d-aeac-2e762809092c-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.773774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.771032    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:08:14.880228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.877756    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-envoy\") pod \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") "
Oct 30 19:08:14.880228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.877817    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-certs\") pod \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") "
Oct 30 19:08:14.880228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.877887    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-data\") pod \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") "
Oct 30 19:08:14.880228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.877915    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-credential-socket\") pod \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") "
Oct 30 19:08:14.880228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.877976    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istiod-ca-cert\") pod \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") "
Oct 30 19:08:14.880228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.878083    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-token\" (UniqueName: \"kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-token\") pod \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") "
Oct 30 19:08:14.880228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.878145    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-podinfo\") pod \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") "
Oct 30 19:08:14.880228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.878188    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-kube-api-access-d4s5g\" (UniqueName: \"kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-kube-api-access-d4s5g\") pod \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") "
Oct 30 19:08:14.880228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.878236    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-socket\") pod \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\" (UID: \"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73\") "
Oct 30 19:08:14.880228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.879852    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-data" pod "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" (UID: "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:14.885587 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.881867    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-socket" pod "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" (UID: "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:14.885587 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.882724    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-certs" pod "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" (UID: "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:14.885587 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.883411    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-credential-socket" pod "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" (UID: "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:14.885587 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.883755    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istiod-ca-cert" pod "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" (UID: "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:14.888006 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.886576    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-podinfo" pod "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" (UID: "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:08:14.888006 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.887663    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-envoy" pod "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" (UID: "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:14.888912 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.888873    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-kube-api-access-d4s5g" pod "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" (UID: "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73"). InnerVolumeSpecName "kube-api-access-d4s5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:14.889652 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.889592    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:14.890432 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.890395    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-token" pod "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" (UID: "8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:14.981945 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.981892    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.982255 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.982232    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.982380 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.982366    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.982477 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.982462    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.982567 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.982555    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.982656 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.982644    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.982762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.982749    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.982865 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.982850    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4s5g\" (UniqueName: \"kubernetes.io/projected/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-kube-api-access-d4s5g\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:14.982957 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:14.982945    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:15.254265 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.254226    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"]
Oct 30 19:08:15.255162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.255141    4925 cpu_manager.go:414] "containerMap: removing container" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy"
Oct 30 19:08:15.255162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.255163    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy"
Oct 30 19:08:15.255348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.255245    4925 cpu_manager.go:414] "containerMap: removing container" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy"
Oct 30 19:08:15.255348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.255253    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy"
Oct 30 19:08:15.255485 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.255470    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" containerName="istio-proxy"
Oct 30 19:08:15.255549 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.255505    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" containerName="istio-proxy"
Oct 30 19:08:15.259523 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.259502    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"
Oct 30 19:08:15.267425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.266462    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-28c1e9-dockercfg-f5p8w\""
Oct 30 19:08:15.267425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.266537    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-28c1e9\""
Oct 30 19:08:15.270081 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.270047    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"]
Oct 30 19:08:15.286204 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.286171    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:15.311233 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.311212    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"]
Oct 30 19:08:15.316346 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.316322    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"
Oct 30 19:08:15.329783 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.329758    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"]
Oct 30 19:08:15.353398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.353376    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"
Oct 30 19:08:15.353543 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.353444    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg" event={"ID":"8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73","Type":"ContainerDied","Data":"814cfa65a290bca433d42a30b52309e1ee76172f53ff5b817332f23e31fb83ae"}
Oct 30 19:08:15.353543 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.353504    4925 scope.go:122] "RemoveContainer" containerID="14dffd84df729d6f05a07c47f828692bfbcf3b7e62002646d744cc98c79678ed"
Oct 30 19:08:15.355687 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.355661    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s" event={"ID":"86c005ba-9d2d-4d5d-aeac-2e762809092c","Type":"ContainerDied","Data":"accbe5d85bbe844478b680c45fbea5a4229a1ae302adff4b3a6f2aec39b78def"}
Oct 30 19:08:15.355922 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.355688    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"
Oct 30 19:08:15.370019 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.370000    4925 scope.go:122] "RemoveContainer" containerID="f6ca2dbc120b4727fa87611a6d1cb758d76ad796ff65404298c2c83705da832d"
Oct 30 19:08:15.392346 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.391011    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4\" (UID: \"0fbcbf42-c4b7-4531-a941-2be9622bd9d5\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"
Oct 30 19:08:15.392487 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.392470    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbcpb\" (UniqueName: \"kubernetes.io/projected/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-kube-api-access-dbcpb\") pod \"payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4\" (UID: \"0fbcbf42-c4b7-4531-a941-2be9622bd9d5\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"
Oct 30 19:08:15.404698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.404679    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"]
Oct 30 19:08:15.415446 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.413833    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-l954s"]
Oct 30 19:08:15.440180 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.434872    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"]
Oct 30 19:08:15.440180 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.436947    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-gz2dg"]
Oct 30 19:08:15.499213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.497642    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbcpb\" (UniqueName: \"kubernetes.io/projected/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-kube-api-access-dbcpb\") pod \"payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4\" (UID: \"0fbcbf42-c4b7-4531-a941-2be9622bd9d5\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"
Oct 30 19:08:15.499213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.497985    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4nwl\" (UniqueName: \"kubernetes.io/projected/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-kube-api-access-x4nwl\") pod \"payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l\" (UID: \"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"
Oct 30 19:08:15.499213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.498192    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l\" (UID: \"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"
Oct 30 19:08:15.499213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.498315    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4\" (UID: \"0fbcbf42-c4b7-4531-a941-2be9622bd9d5\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"
Oct 30 19:08:15.503761 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.503726    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:15.525501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.524039    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4\" (UID: \"0fbcbf42-c4b7-4531-a941-2be9622bd9d5\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"
Oct 30 19:08:15.596085 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.596020    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbcpb\" (UniqueName: \"kubernetes.io/projected/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-kube-api-access-dbcpb\") pod \"payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4\" (UID: \"0fbcbf42-c4b7-4531-a941-2be9622bd9d5\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"
Oct 30 19:08:15.604672 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.604399    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"
Oct 30 19:08:15.680293 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.679122    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4nwl\" (UniqueName: \"kubernetes.io/projected/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-kube-api-access-x4nwl\") pod \"payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l\" (UID: \"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"
Oct 30 19:08:15.680718 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.680684    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l\" (UID: \"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"
Oct 30 19:08:15.682056 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.682032    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l\" (UID: \"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"
Oct 30 19:08:15.727742 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.720658    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4nwl\" (UniqueName: \"kubernetes.io/projected/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-kube-api-access-x4nwl\") pod \"payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l\" (UID: \"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"
Oct 30 19:08:15.925008 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.924936    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:15.936415 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.936386    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"]
Oct 30 19:08:15.943883 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.943860    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:15.949043 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.949007    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-4a477d\"/\"payload-processing-external-model-plugins-e2e-extmod-b-4a477d\""
Oct 30 19:08:15.949201 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.949165    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-4a477d\"/\"agc-cred-bd948f818f73\""
Oct 30 19:08:15.950040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.949354    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-4a477d\"/\"payload-processing-external-model-e2e--27d7d667-dockercfg-ffnd8\""
Oct 30 19:08:15.950040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.949460    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-4a477d\"/\"openshift-service-ca.crt\""
Oct 30 19:08:15.950040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.949604    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-4a477d\"/\"kube-root-ca.crt\""
Oct 30 19:08:15.950040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.949694    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-4a477d\"/\"routing-overlay\""
Oct 30 19:08:15.951011 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.950991    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"
Oct 30 19:08:15.955792 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:15.955721    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"]
Oct 30 19:08:16.005549 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.005186    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzdfc\" (UniqueName: \"kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-kube-api-access-pzdfc\") pod \"payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") " pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.005549 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.005226    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") " pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.005549 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.005256    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") " pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.005549 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.005379    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") " pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.135247 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.108254    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") " pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.135247 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.108497    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzdfc\" (UniqueName: \"kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-kube-api-access-pzdfc\") pod \"payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") " pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.135247 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.108532    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") " pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.135247 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.108570    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") " pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.135247 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.109944    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") " pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.135247 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.132555    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") " pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.135247 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.133422    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") " pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.138119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.137725    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzdfc\" (UniqueName: \"kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-kube-api-access-pzdfc\") pod \"payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") " pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.280752 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.280444    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:16.281804 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.281774    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"
Oct 30 19:08:16.387564 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.387161    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"]
Oct 30 19:08:16.455912 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.455882    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86c005ba-9d2d-4d5d-aeac-2e762809092c" path="/var/lib/kubelet/pods/86c005ba-9d2d-4d5d-aeac-2e762809092c/volumes"
Oct 30 19:08:16.481249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.476529    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73" path="/var/lib/kubelet/pods/8cb8bc37-2bab-4541-a4e0-e3bc9fb47d73/volumes"
Oct 30 19:08:16.481249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.478324    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4" event={"ID":"0fbcbf42-c4b7-4531-a941-2be9622bd9d5","Type":"ContainerStarted","Data":"b2d7bb40a8d705968302f80f84abc34e545944b2e4505325c343f06379205491"}
Oct 30 19:08:16.639599 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.639506    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:16.760521 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.760490    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"]
Oct 30 19:08:16.886887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:16.886855    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"]
Oct 30 19:08:16.887222 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:08:16.887200    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22c9df2a_2a2d_4127_af5d_1f82c7af9726.slice/crio-08d381e719ef3a69a0e0d9d1e00f130db4c358aeef1f042e91c7f1cc23d6b9e2 WatchSource:0}: Error finding container 08d381e719ef3a69a0e0d9d1e00f130db4c358aeef1f042e91c7f1cc23d6b9e2: Status 404 returned error can't find the container with id 08d381e719ef3a69a0e0d9d1e00f130db4c358aeef1f042e91c7f1cc23d6b9e2
Oct 30 19:08:17.321171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:17.312767    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:17.469084 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:17.469047    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2" event={"ID":"22c9df2a-2a2d-4127-af5d-1f82c7af9726","Type":"ContainerStarted","Data":"cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73"}
Oct 30 19:08:17.469292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:17.469143    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2" event={"ID":"22c9df2a-2a2d-4127-af5d-1f82c7af9726","Type":"ContainerStarted","Data":"08d381e719ef3a69a0e0d9d1e00f130db4c358aeef1f042e91c7f1cc23d6b9e2"}
Oct 30 19:08:17.469292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:17.469211    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:17.471264 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:17.471231    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4" event={"ID":"0fbcbf42-c4b7-4531-a941-2be9622bd9d5","Type":"ContainerStarted","Data":"8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7"}
Oct 30 19:08:17.471420 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:17.471330    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"
Oct 30 19:08:17.473261 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:17.473241    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l" event={"ID":"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c","Type":"ContainerStarted","Data":"525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5"}
Oct 30 19:08:17.473385 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:17.473276    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l" event={"ID":"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c","Type":"ContainerStarted","Data":"c78323aab6cb801160ae81ae1dfc3fcdb7cd15e40a526335ed1f3b9858caf970"}
Oct 30 19:08:17.473385 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:17.473297    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"
Oct 30 19:08:17.494837 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:17.494784    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2" podStartSLOduration=2.494768255 podStartE2EDuration="2.494768255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:17.487473308 +0000 UTC m=+1919.203818385" watchObservedRunningTime="2026-10-30 19:08:17.494768255 +0000 UTC m=+1919.211113334"
Oct 30 19:08:17.515524 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:17.515484    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l" podStartSLOduration=2.515473167 podStartE2EDuration="2.515473167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:17.511653514 +0000 UTC m=+1919.227998599" watchObservedRunningTime="2026-10-30 19:08:17.515473167 +0000 UTC m=+1919.231818278"
Oct 30 19:08:17.536136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:17.536073    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4" podStartSLOduration=2.536058406 podStartE2EDuration="2.536058406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:17.533520582 +0000 UTC m=+1919.249865661" watchObservedRunningTime="2026-10-30 19:08:17.536058406 +0000 UTC m=+1919.252403477"
Oct 30 19:08:18.113947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:18.113916    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:18.782126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:18.773070    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:19.566538 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:19.566477    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:20.361726 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:20.361696    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:08:20.886879 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:20.886777    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:21.493965 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.493930    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:21.795501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.795411    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"]
Oct 30 19:08:21.802565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.802541    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.827642 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.827615    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"]
Oct 30 19:08:21.886353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.886319    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-istio-token\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.886513 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.886389    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/380c3494-b8be-4efd-b495-414a2819d883-istiod-ca-cert\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.886513 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.886482    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.886639 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.886539    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4hr5\" (UniqueName: \"kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-kube-api-access-w4hr5\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.886699 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.886663    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-envoy\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.886790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.886769    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-data\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.886901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.886884    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-credential-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.887286 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.887021    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-certs\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.887286 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.887114    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/380c3494-b8be-4efd-b495-414a2819d883-istio-podinfo\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.889544 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.889513    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-f3d6a5-openshift-default-dockercfg-8dmt5\""
Oct 30 19:08:21.988485 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.988443    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-istio-token\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.988691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.988499    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/380c3494-b8be-4efd-b495-414a2819d883-istiod-ca-cert\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.988691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.988553    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.988691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.988612    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4hr5\" (UniqueName: \"kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-kube-api-access-w4hr5\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.988882 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.988689    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-envoy\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.988882 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.988872    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-data\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.989227 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.989015    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.989348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.989321    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/380c3494-b8be-4efd-b495-414a2819d883-istiod-ca-cert\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.990141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.990075    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-credential-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.990286 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.990260    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-certs\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.990372 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.990348    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-data\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.990372 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.990354    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/380c3494-b8be-4efd-b495-414a2819d883-istio-podinfo\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.990492 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.990355    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-credential-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.990851 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.990827    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-certs\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.991694 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.991669    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-envoy\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.994726 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.994704    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/380c3494-b8be-4efd-b495-414a2819d883-istio-podinfo\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.997631 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.997609    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-istio-token\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:21.997718 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:21.997643    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4hr5\" (UniqueName: \"kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-kube-api-access-w4hr5\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:22.123199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.123126    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"]
Oct 30 19:08:22.132140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.131773    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.136906 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.135703    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:22.141141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.137585    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"]
Oct 30 19:08:22.141141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.138650    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-c164d6-serving-cert\""
Oct 30 19:08:22.337407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.269780    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:22.337407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.295445    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ddb6fef-cf25-4773-b561-eea818ebc218-openshift-service-ca\") pod \"maas-api-e2e-autores-c164d6-5456d697d7-5tsgf\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.337407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.295657    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-maas-api-tls\") pod \"maas-api-e2e-autores-c164d6-5456d697d7-5tsgf\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.337407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.295784    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-metrics-certs\") pod \"maas-api-e2e-autores-c164d6-5456d697d7-5tsgf\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.337407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.295839    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvhh7\" (UniqueName: \"kubernetes.io/projected/7ddb6fef-cf25-4773-b561-eea818ebc218-kube-api-access-zvhh7\") pod \"maas-api-e2e-autores-c164d6-5456d697d7-5tsgf\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.398037 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.397167    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ddb6fef-cf25-4773-b561-eea818ebc218-openshift-service-ca\") pod \"maas-api-e2e-autores-c164d6-5456d697d7-5tsgf\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.398037 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.397360    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-maas-api-tls\") pod \"maas-api-e2e-autores-c164d6-5456d697d7-5tsgf\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.398037 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.397485    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-metrics-certs\") pod \"maas-api-e2e-autores-c164d6-5456d697d7-5tsgf\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.398037 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.397533    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvhh7\" (UniqueName: \"kubernetes.io/projected/7ddb6fef-cf25-4773-b561-eea818ebc218-kube-api-access-zvhh7\") pod \"maas-api-e2e-autores-c164d6-5456d697d7-5tsgf\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.398774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.398679    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ddb6fef-cf25-4773-b561-eea818ebc218-openshift-service-ca\") pod \"maas-api-e2e-autores-c164d6-5456d697d7-5tsgf\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.406016 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.405990    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-metrics-certs\") pod \"maas-api-e2e-autores-c164d6-5456d697d7-5tsgf\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.412613 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.412580    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvhh7\" (UniqueName: \"kubernetes.io/projected/7ddb6fef-cf25-4773-b561-eea818ebc218-kube-api-access-zvhh7\") pod \"maas-api-e2e-autores-c164d6-5456d697d7-5tsgf\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.412818 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.412797    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-maas-api-tls\") pod \"maas-api-e2e-autores-c164d6-5456d697d7-5tsgf\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.455144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.455068    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:22.471613 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.471590    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"]
Oct 30 19:08:22.481278 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:08:22.481251    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod380c3494_b8be_4efd_b495_414a2819d883.slice/crio-cef4eb09042f6f01fcbfe25f8e1e252a8dbba7329d83e5be5afd89e09cad0631 WatchSource:0}: Error finding container cef4eb09042f6f01fcbfe25f8e1e252a8dbba7329d83e5be5afd89e09cad0631: Status 404 returned error can't find the container with id cef4eb09042f6f01fcbfe25f8e1e252a8dbba7329d83e5be5afd89e09cad0631
Oct 30 19:08:22.490482 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.490448    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:08:22.491997 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.491966    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:08:22.492232 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.492192    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:08:22.502466 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.502282    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:22.541268 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.541203    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h" event={"ID":"380c3494-b8be-4efd-b495-414a2819d883","Type":"ContainerStarted","Data":"cef4eb09042f6f01fcbfe25f8e1e252a8dbba7329d83e5be5afd89e09cad0631"}
Oct 30 19:08:22.661257 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.653481    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:22.856924 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.854264    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"]
Oct 30 19:08:22.894164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.889363    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:22.900542 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.900400    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-baa504-openshift-default-dockercfg-tbb42\""
Oct 30 19:08:22.920197 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.917340    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"]
Oct 30 19:08:22.930770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.930736    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-envoy\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:22.931171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.930858    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4469a92-c400-468a-8309-13f96c4509dc-istio-podinfo\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:22.931171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.930914    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-data\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:22.931171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.930958    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-certs\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:22.931171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.931046    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-socket\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:22.931171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.931122    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-credential-socket\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:22.931171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.931156    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgz2f\" (UniqueName: \"kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-kube-api-access-wgz2f\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:22.931514 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.931270    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-istio-token\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:22.931514 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.931374    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4469a92-c400-468a-8309-13f96c4509dc-istiod-ca-cert\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:22.948297 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:22.947771    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"]
Oct 30 19:08:23.032559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.032524    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-envoy\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.032786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.032599    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4469a92-c400-468a-8309-13f96c4509dc-istio-podinfo\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.032786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.032631    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-data\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.032786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.032656    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-certs\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.032786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.032715    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-socket\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.033022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.032819    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-credential-socket\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.033022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.032866    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgz2f\" (UniqueName: \"kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-kube-api-access-wgz2f\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.033022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.032986    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-istio-token\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.033207 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.033073    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4469a92-c400-468a-8309-13f96c4509dc-istiod-ca-cert\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.033945 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.033920    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4469a92-c400-468a-8309-13f96c4509dc-istiod-ca-cert\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.034277 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.034253    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-data\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.034354 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.034274    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-certs\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.035961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.035470    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-socket\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.035961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.035542    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-credential-socket\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.035961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.035846    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-envoy\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.035961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.035903    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4469a92-c400-468a-8309-13f96c4509dc-istio-podinfo\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.041346 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.041318    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgz2f\" (UniqueName: \"kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-kube-api-access-wgz2f\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.041691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.041669    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-istio-token\") pod \"e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") " pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.222529 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.222430    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:23.256563 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.256528    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:23.529171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.529139    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"]
Oct 30 19:08:23.536035 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.536005    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"]
Oct 30 19:08:23.536234 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.536144    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.581777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.578775    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-e51c10-openshift-default-dockercfg-s9hfx\""
Oct 30 19:08:23.615133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.608909    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"]
Oct 30 19:08:23.615133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.613545    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf" event={"ID":"7ddb6fef-cf25-4773-b561-eea818ebc218","Type":"ContainerStarted","Data":"36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f"}
Oct 30 19:08:23.615133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.614197    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf" event={"ID":"7ddb6fef-cf25-4773-b561-eea818ebc218","Type":"ContainerStarted","Data":"2651a5fcdc815ce5a955ddcf6bd0539f10b85f5286a53e896c0562449c9d2a22"}
Oct 30 19:08:23.615133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.614278    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:23.619870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.619053    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h" event={"ID":"380c3494-b8be-4efd-b495-414a2819d883","Type":"ContainerStarted","Data":"4740448450e61855e5c0676c85b49eb968b1c05cacfa6499201de2d7215a81ca"}
Oct 30 19:08:23.619994 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.619890    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:08:23.619994 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.619971    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:08:23.620073 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.620019    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:08:23.641534 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.641488    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf" podStartSLOduration=1.188412485 podStartE2EDuration="1.641474554s" totalImagesPullingTime="453.062069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:23.633855934 +0000 UTC m=+1925.350201013" watchObservedRunningTime="2026-10-30 19:08:23.641474554 +0000 UTC m=+1925.357819646"
Oct 30 19:08:23.653837 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.653811    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-data\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.654026 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.654005    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-certs\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.654200 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.654181    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-socket\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.654367 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.654348    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istiod-ca-cert\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.654486 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.654474    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2npd\" (UniqueName: \"kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-kube-api-access-k2npd\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.654605 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.654588    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-token\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.654721 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.654710    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-credential-socket\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.654816 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.654807    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-envoy\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.654990 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.654962    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-podinfo\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.665127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.658768    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h" podStartSLOduration=2.65875021 podStartE2EDuration="2.65875021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:23.655971548 +0000 UTC m=+1925.372316612" watchObservedRunningTime="2026-10-30 19:08:23.65875021 +0000 UTC m=+1925.375095289"
Oct 30 19:08:23.762128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.761455    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-data\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.762128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.761511    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-certs\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.762128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.761547    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-socket\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.762128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.761607    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istiod-ca-cert\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.762128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.761668    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2npd\" (UniqueName: \"kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-kube-api-access-k2npd\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.762128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.761743    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-token\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.762128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.761788    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-credential-socket\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.762128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.761827    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-envoy\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.762128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.761890    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-podinfo\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.762764 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.762580    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-certs\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.763931 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.763403    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-data\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.763931 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.763654    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-credential-socket\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.763931 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.763883    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-socket\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.782342 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.764767    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istiod-ca-cert\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.803124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.786665    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-podinfo\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.829237 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.821040    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-token\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.829237 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.821500    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-envoy\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.829237 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.824407    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2npd\" (UniqueName: \"kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-kube-api-access-k2npd\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:23.908531 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:23.908494    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:24.152948 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.151052    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:24.158670 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.156936    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.222:15021/healthz/ready\": dial tcp 10.132.0.222:15021: connect: connection refused" start-of-body=
Oct 30 19:08:24.158670 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.156990    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h" podUID="380c3494-b8be-4efd-b495-414a2819d883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.222:15021/healthz/ready\": dial tcp 10.132.0.222:15021: connect: connection refused"
Oct 30 19:08:24.158670 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.158158    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:24.405189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.401503    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"]
Oct 30 19:08:24.407738 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:08:24.407714    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b1140b_4cbd_49b0_921f_bbcc2986ebb5.slice/crio-dacb012feb6aefa5f11e41dbe9b0023037628a8850e2241f0710dc3be3b2b97a WatchSource:0}: Error finding container dacb012feb6aefa5f11e41dbe9b0023037628a8850e2241f0710dc3be3b2b97a: Status 404 returned error can't find the container with id dacb012feb6aefa5f11e41dbe9b0023037628a8850e2241f0710dc3be3b2b97a
Oct 30 19:08:24.413554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.413497    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:08:24.413660 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.413634    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:08:24.413714 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.413698    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:08:24.629616 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.629581    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" event={"ID":"e4469a92-c400-468a-8309-13f96c4509dc","Type":"ContainerStarted","Data":"bc4488f827c10ecf28a470a1d5f333b96968f1cad11abd24e88d28e6871a9ef2"}
Oct 30 19:08:24.629890 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.629855    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" event={"ID":"e4469a92-c400-468a-8309-13f96c4509dc","Type":"ContainerStarted","Data":"220bcf64c4729afc9420ba862d93e3327eceae997be96865868580d665ce895b"}
Oct 30 19:08:24.631837 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.631808    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7" event={"ID":"33b1140b-4cbd-49b0-921f-bbcc2986ebb5","Type":"ContainerStarted","Data":"90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47"}
Oct 30 19:08:24.631950 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.631895    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7" event={"ID":"33b1140b-4cbd-49b0-921f-bbcc2986ebb5","Type":"ContainerStarted","Data":"dacb012feb6aefa5f11e41dbe9b0023037628a8850e2241f0710dc3be3b2b97a"}
Oct 30 19:08:24.702493 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.702430    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" podStartSLOduration=2.702407399 podStartE2EDuration="2.702407399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:24.697388507 +0000 UTC m=+1926.413733586" watchObservedRunningTime="2026-10-30 19:08:24.702407399 +0000 UTC m=+1926.418752478"
Oct 30 19:08:24.921123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:24.911179    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:25.138338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:25.138223    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.222:15021/healthz/ready\": dial tcp 10.132.0.222:15021: connect: connection refused" start-of-body=
Oct 30 19:08:25.138338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:25.138273    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h" podUID="380c3494-b8be-4efd-b495-414a2819d883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.222:15021/healthz/ready\": dial tcp 10.132.0.222:15021: connect: connection refused"
Oct 30 19:08:25.258120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:25.257468    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:25.266316 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:25.266267    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused" start-of-body=
Oct 30 19:08:25.266638 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:25.266589    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" podUID="e4469a92-c400-468a-8309-13f96c4509dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused"
Oct 30 19:08:25.318653 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:25.318616    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:25.912118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:25.911662    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.225:15021/healthz/ready\": dial tcp 10.132.0.225:15021: connect: connection refused" start-of-body=
Oct 30 19:08:25.912118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:25.911727    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7" podUID="33b1140b-4cbd-49b0-921f-bbcc2986ebb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.225:15021/healthz/ready\": dial tcp 10.132.0.225:15021: connect: connection refused"
Oct 30 19:08:26.163174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:26.162974    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.222:15021/healthz/ready\": dial tcp 10.132.0.222:15021: connect: connection refused" start-of-body=
Oct 30 19:08:26.163174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:26.163033    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h" podUID="380c3494-b8be-4efd-b495-414a2819d883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.222:15021/healthz/ready\": dial tcp 10.132.0.222:15021: connect: connection refused"
Oct 30 19:08:26.272125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:26.266914    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused" start-of-body=
Oct 30 19:08:26.272125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:26.266971    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" podUID="e4469a92-c400-468a-8309-13f96c4509dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused"
Oct 30 19:08:26.847866 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:26.847827    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:26.917616 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:26.917575    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.225:15021/healthz/ready\": dial tcp 10.132.0.225:15021: connect: connection refused" start-of-body=
Oct 30 19:08:26.917916 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:26.917883    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7" podUID="33b1140b-4cbd-49b0-921f-bbcc2986ebb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.225:15021/healthz/ready\": dial tcp 10.132.0.225:15021: connect: connection refused"
Oct 30 19:08:27.146842 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:27.146239    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.222:15021/healthz/ready\": dial tcp 10.132.0.222:15021: connect: connection refused" start-of-body=
Oct 30 19:08:27.146842 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:27.146318    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h" podUID="380c3494-b8be-4efd-b495-414a2819d883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.222:15021/healthz/ready\": dial tcp 10.132.0.222:15021: connect: connection refused"
Oct 30 19:08:27.257920 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:27.257481    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused" start-of-body=
Oct 30 19:08:27.257920 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:27.257547    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" podUID="e4469a92-c400-468a-8309-13f96c4509dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused"
Oct 30 19:08:27.917124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:27.914191    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.225:15021/healthz/ready\": dial tcp 10.132.0.225:15021: connect: connection refused" start-of-body=
Oct 30 19:08:27.917124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:27.914261    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7" podUID="33b1140b-4cbd-49b0-921f-bbcc2986ebb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.225:15021/healthz/ready\": dial tcp 10.132.0.225:15021: connect: connection refused"
Oct 30 19:08:28.170206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:28.155241    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.222:15021/healthz/ready\": dial tcp 10.132.0.222:15021: connect: connection refused" start-of-body=
Oct 30 19:08:28.170206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:28.155301    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h" podUID="380c3494-b8be-4efd-b495-414a2819d883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.222:15021/healthz/ready\": dial tcp 10.132.0.222:15021: connect: connection refused"
Oct 30 19:08:28.273139 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:28.264845    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused" start-of-body=
Oct 30 19:08:28.273139 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:28.264908    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" podUID="e4469a92-c400-468a-8309-13f96c4509dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused"
Oct 30 19:08:28.362121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:28.358617    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:28.640058 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:28.639909    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"
Oct 30 19:08:28.645749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:28.640324    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"
Oct 30 19:08:28.645749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:28.640389    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:08:28.910172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:28.886943    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7" podStartSLOduration=5.886920621 podStartE2EDuration="5.886920621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:24.784669465 +0000 UTC m=+1926.501014545" watchObservedRunningTime="2026-10-30 19:08:28.886920621 +0000 UTC m=+1930.603265698"
Oct 30 19:08:28.932149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:28.924461    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.225:15021/healthz/ready\": dial tcp 10.132.0.225:15021: connect: connection refused" start-of-body=
Oct 30 19:08:28.932149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:28.924525    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7" podUID="33b1140b-4cbd-49b0-921f-bbcc2986ebb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.225:15021/healthz/ready\": dial tcp 10.132.0.225:15021: connect: connection refused"
Oct 30 19:08:29.283171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:29.283065    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.222:15021/healthz/ready\": dial tcp 10.132.0.222:15021: connect: connection refused" start-of-body=
Oct 30 19:08:29.283742 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:29.283708    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h" podUID="380c3494-b8be-4efd-b495-414a2819d883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.222:15021/healthz/ready\": dial tcp 10.132.0.222:15021: connect: connection refused"
Oct 30 19:08:29.309123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:29.285312    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused" start-of-body=
Oct 30 19:08:29.309407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:29.309369    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" podUID="e4469a92-c400-468a-8309-13f96c4509dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused"
Oct 30 19:08:29.537186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:29.536921    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded"
Oct 30 19:08:29.590175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:29.564209    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:08:29.622312 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:29.621230    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:08:29.692438 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:29.692407    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:08:29.940380 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:29.921837    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.225:15021/healthz/ready\": dial tcp 10.132.0.225:15021: connect: connection refused" start-of-body=
Oct 30 19:08:29.940380 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:29.921955    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7" podUID="33b1140b-4cbd-49b0-921f-bbcc2986ebb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.225:15021/healthz/ready\": dial tcp 10.132.0.225:15021: connect: connection refused"
Oct 30 19:08:30.066122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:30.042924    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:30.245175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:30.244662    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:30.249123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:30.245772    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:30.249901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:30.249857    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:08:30.256842 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:30.256815    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused" start-of-body=
Oct 30 19:08:30.262253 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:30.262223    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" podUID="e4469a92-c400-468a-8309-13f96c4509dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused"
Oct 30 19:08:30.910339 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:30.910297    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.225:15021/healthz/ready\": dial tcp 10.132.0.225:15021: connect: connection refused" start-of-body=
Oct 30 19:08:30.911377 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:30.911340    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7" podUID="33b1140b-4cbd-49b0-921f-bbcc2986ebb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.225:15021/healthz/ready\": dial tcp 10.132.0.225:15021: connect: connection refused"
Oct 30 19:08:31.267652 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:31.267563    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused" start-of-body=
Oct 30 19:08:31.267922 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:31.267891    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" podUID="e4469a92-c400-468a-8309-13f96c4509dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.224:15021/healthz/ready\": dial tcp 10.132.0.224:15021: connect: connection refused"
Oct 30 19:08:31.318123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:31.317921    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:31.914698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:31.914663    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:31.915153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:31.915132    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:31.915884 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:31.915861    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:08:32.170548 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:32.170459    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:32.262639 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:32.262605    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:32.263135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:32.263110    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:32.263895 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:32.263872    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:08:32.916088 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:32.916047    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:33.722119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:33.714349    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:34.582788 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:34.582735    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:35.538361 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:35.538322    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:37.400488 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:37.400450    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:37.594143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:37.594074    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:37.909608 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:37.909574    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:38.599124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:38.596132    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:39.462991 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:39.462925    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:40.101028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:40.100990    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:40.398174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:40.397744    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:41.019274 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:41.019239    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:41.633607 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:41.633575    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:41.938121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:41.929190    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:42.200187 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:42.188261    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:42.414903 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:42.414866    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:42.615910 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:42.615823    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:42.953444 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:42.953403    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:43.253826 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:43.253743    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:43.346545 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:08:43.346504    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:08:43.682658 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:43.682053    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:43.872655 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:43.872603    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:44.761977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:44.761876    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:45.570306 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:45.570260    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:46.318120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:46.317779    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:47.015428 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:47.014997    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:48.058760 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:48.058725    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:48.874321 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:48.874283    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:49.616976 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:49.616939    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:50.311344 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:50.311297    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:51.198550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:51.198496    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:51.886156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:51.886087    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:52.578173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:52.577825    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:52.784754 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:52.783983    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:53.333122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:53.328055    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:54.071140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:54.067965    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:54.682466 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:54.682430    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:55.463789 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:55.463750    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:55.661124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:55.660210    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:56.363766 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:56.363712    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:57.007641 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:57.007604    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:59.165785 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:59.165748    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:59.491727 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:59.491640    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:59.726410 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:08:59.726375    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:00.145783 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:00.145746    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:00.891915 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:00.891881    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:01.139856 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:01.139824    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:01.375199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:01.375037    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:01.587766 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:01.587730    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:01.836661 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:01.836627    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:02.352689 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:02.352651    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:02.975976 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:02.975925    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:03.409734 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:03.409693    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:04.139556 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:04.139510    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:04.938662 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:04.938621    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:05.748693 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:05.748657    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:06.582119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:06.581366    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:06.905280 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:06.905196    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"]
Oct 30 19:09:06.919490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:06.919465    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"]
Oct 30 19:09:06.919869 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:06.919619    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:06.929330 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:06.929299    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"]
Oct 30 19:09:06.929471 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:06.929414    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:06.931591 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:06.931563    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"]
Oct 30 19:09:07.025778 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.025750    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-certs\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.025953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.025789    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e4a918-26ba-4ebd-8822-a182e8c84418-istiod-ca-cert\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.025953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.025818    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-podinfo\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.025953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.025914    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf268\" (UniqueName: \"kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-kube-api-access-qf268\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.025953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.025943    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-podinfo\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.026151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.025961    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.026151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.026065    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-token\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.026246 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.026157    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istiod-ca-cert\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.026246 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.026186    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-envoy\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.026321 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.026271    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-token\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.026368 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.026317    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gwj6\" (UniqueName: \"kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-kube-api-access-7gwj6\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.026368 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.026344    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-data\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.026442 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.026402    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-credential-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.026442 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.026436    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-certs\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.026590 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.026572    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-envoy\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.026717 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.026689    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-data\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.026793 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.026775    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-credential-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.026839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.026823    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.127787 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.127751    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-credential-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.127973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.127794    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.127973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.127820    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-certs\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.127973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.127849    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e4a918-26ba-4ebd-8822-a182e8c84418-istiod-ca-cert\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.127973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.127878    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-podinfo\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.127973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.127942    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf268\" (UniqueName: \"kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-kube-api-access-qf268\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.128218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.127981    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-podinfo\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.128218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128011    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.128218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128072    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-token\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.128218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128143    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istiod-ca-cert\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.128218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128168    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-envoy\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.128412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128229    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-token\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.128412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128247    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.128412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128268    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-credential-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.128412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128333    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-certs\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.128693 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128666    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e4a918-26ba-4ebd-8822-a182e8c84418-istiod-ca-cert\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.128824 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128740    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gwj6\" (UniqueName: \"kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-kube-api-access-7gwj6\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.128824 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128673    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.128824 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128783    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-data\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.129151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128841    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-credential-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.129151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.128891    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-certs\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.129151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.129001    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-envoy\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.129151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.129126    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-data\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.129389 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.129159    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-data\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.129389 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.129198    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istiod-ca-cert\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.129389 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.129259    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-credential-socket\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.129557 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.129497    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-certs\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.129644 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.129607    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-data\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.131134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.131087    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-envoy\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.131259 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.131133    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-podinfo\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.131466 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.131446    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-podinfo\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.131771 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.131754    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-envoy\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.138457 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.138428    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-token\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.138574 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.138509    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gwj6\" (UniqueName: \"kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-kube-api-access-7gwj6\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.138892 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.138871    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-token\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.138892 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.138882    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf268\" (UniqueName: \"kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-kube-api-access-qf268\") pod \"e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") " pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.259918 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.257529    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:07.267233 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.266995    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:07.316686 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.316630    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:07.661499 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:09:07.661461    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod227555e2_ac4e_4033_a23b_0c43c6eb37ac.slice/crio-4c1b2a693b1896bf4b996c1121570ef7aa6ddfd1f8a0f1774010affa30c2944d WatchSource:0}: Error finding container 4c1b2a693b1896bf4b996c1121570ef7aa6ddfd1f8a0f1774010affa30c2944d: Status 404 returned error can't find the container with id 4c1b2a693b1896bf4b996c1121570ef7aa6ddfd1f8a0f1774010affa30c2944d
Oct 30 19:09:07.663703 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.663681    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"]
Oct 30 19:09:07.665329 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.665288    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:07.665436 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.665375    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:07.665436 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.665417    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:07.671271 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.671242    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"]
Oct 30 19:09:07.688300 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:09:07.688271    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e4a918_26ba_4ebd_8822_a182e8c84418.slice/crio-8d60d16ac7f9a09104b877eb0c4e0e55de639a4b3946e208a15404ebaae6be0e WatchSource:0}: Error finding container 8d60d16ac7f9a09104b877eb0c4e0e55de639a4b3946e208a15404ebaae6be0e: Status 404 returned error can't find the container with id 8d60d16ac7f9a09104b877eb0c4e0e55de639a4b3946e208a15404ebaae6be0e
Oct 30 19:09:07.692068 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.692033    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:07.692191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.692139    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:07.692191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:07.692184    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:08.129131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.121409    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:08.200645 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.200598    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" event={"ID":"67e4a918-26ba-4ebd-8822-a182e8c84418","Type":"ContainerStarted","Data":"bcf6dff602e924504b405ee69603368a5cecb943ddb22666a0e762646d4a5090"}
Oct 30 19:09:08.200939 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.200900    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" event={"ID":"67e4a918-26ba-4ebd-8822-a182e8c84418","Type":"ContainerStarted","Data":"8d60d16ac7f9a09104b877eb0c4e0e55de639a4b3946e208a15404ebaae6be0e"}
Oct 30 19:09:08.208891 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.208858    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" event={"ID":"227555e2-ac4e-4033-a23b-0c43c6eb37ac","Type":"ContainerStarted","Data":"824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c"}
Oct 30 19:09:08.209192 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.209168    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" event={"ID":"227555e2-ac4e-4033-a23b-0c43c6eb37ac","Type":"ContainerStarted","Data":"4c1b2a693b1896bf4b996c1121570ef7aa6ddfd1f8a0f1774010affa30c2944d"}
Oct 30 19:09:08.239324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.239257    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" podStartSLOduration=2.239240517 podStartE2EDuration="2.239240517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:08.229226078 +0000 UTC m=+1969.945571156" watchObservedRunningTime="2026-10-30 19:09:08.239240517 +0000 UTC m=+1969.955585594"
Oct 30 19:09:08.253576 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.253522    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" podStartSLOduration=2.253506567 podStartE2EDuration="2.253506567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:08.251195021 +0000 UTC m=+1969.967540098" watchObservedRunningTime="2026-10-30 19:09:08.253506567 +0000 UTC m=+1969.969851644"
Oct 30 19:09:08.265148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.259610    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:08.265148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.262052    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused" start-of-body=
Oct 30 19:09:08.265148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.262156    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused"
Oct 30 19:09:08.268168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.268142    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:08.271256 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.271231    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused" start-of-body=
Oct 30 19:09:08.271391 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.271276    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused"
Oct 30 19:09:08.626619 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.626588    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"]
Oct 30 19:09:08.631960 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.631939    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.690249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.690209    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-certs\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.690920 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.690866    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-token\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.691505 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.691211    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-socket\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.691755 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.691734    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b330c3ad-b62a-4146-9626-d8e9045512ff-istiod-ca-cert\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.691926 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.691909    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-podinfo\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.692111 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.692079    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-credential-socket\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.692232 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.692218    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-envoy\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.692401 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.692385    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtm69\" (UniqueName: \"kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-kube-api-access-mtm69\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.692503 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.692491    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-data\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.695677 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.695644    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"]
Oct 30 19:09:08.794007 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.793974    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-credential-socket\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.794218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.794038    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-envoy\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.794218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.794166    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtm69\" (UniqueName: \"kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-kube-api-access-mtm69\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.794218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.794199    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-data\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.794406 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.794325    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-certs\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.794406 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.794355    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-token\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.794602 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.794470    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-credential-socket\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.794782 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.794758    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-data\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.794887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.794765    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-socket\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.794887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.794479    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-socket\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.794887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.794805    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-certs\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.794887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.794881    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b330c3ad-b62a-4146-9626-d8e9045512ff-istiod-ca-cert\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.795119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.794998    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-podinfo\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.795705 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.795604    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b330c3ad-b62a-4146-9626-d8e9045512ff-istiod-ca-cert\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.798481 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.798460    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-envoy\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.798711 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.798677    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-podinfo\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.808886 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.808859    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-token\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.810372 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.810348    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtm69\" (UniqueName: \"kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-kube-api-access-mtm69\") pod \"e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") " pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:08.962126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:08.956500    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:09.371203 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:09.369666    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused" start-of-body=
Oct 30 19:09:09.371203 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:09.369729    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused"
Oct 30 19:09:09.371203 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:09.369807    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused" start-of-body=
Oct 30 19:09:09.371203 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:09.369825    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused"
Oct 30 19:09:09.422121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:09.421584    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:09.636830 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:09.636791    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:09.636948 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:09.636878    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:09.636948 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:09.636922    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:09.638274 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:09.638240    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"]
Oct 30 19:09:10.215228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:10.211899    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:10.330116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:10.328719    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused" start-of-body=
Oct 30 19:09:10.330116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:10.328777    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused"
Oct 30 19:09:10.344119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:10.339429    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused" start-of-body=
Oct 30 19:09:10.344119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:10.339481    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused"
Oct 30 19:09:10.687507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:10.687470    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:10.831586 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:10.831532    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" event={"ID":"b330c3ad-b62a-4146-9626-d8e9045512ff","Type":"ContainerStarted","Data":"d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d"}
Oct 30 19:09:10.831862 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:10.831829    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" event={"ID":"b330c3ad-b62a-4146-9626-d8e9045512ff","Type":"ContainerStarted","Data":"f621c05a1abcd8a5c14f66dad2cd25536574f0d4c5e3ae2afc9567650b7cd326"}
Oct 30 19:09:10.944043 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:10.943933    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" podStartSLOduration=2.943913699 podStartE2EDuration="2.943913699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:10.880845352 +0000 UTC m=+1972.597190429" watchObservedRunningTime="2026-10-30 19:09:10.943913699 +0000 UTC m=+1972.660258778"
Oct 30 19:09:10.979117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:10.962279    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:10.979117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:10.962381    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused" start-of-body=
Oct 30 19:09:10.979117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:10.962453    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" podUID="b330c3ad-b62a-4146-9626-d8e9045512ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused"
Oct 30 19:09:11.024167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:11.017790    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:11.294168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:11.284305    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused" start-of-body=
Oct 30 19:09:11.294168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:11.284368    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused"
Oct 30 19:09:11.294168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:11.284800    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused" start-of-body=
Oct 30 19:09:11.294168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:11.284842    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused"
Oct 30 19:09:11.970135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:11.963507    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused" start-of-body=
Oct 30 19:09:11.970135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:11.963570    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" podUID="b330c3ad-b62a-4146-9626-d8e9045512ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused"
Oct 30 19:09:12.255178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:12.255074    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:12.275428 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:12.259484    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused" start-of-body=
Oct 30 19:09:12.275742 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:12.275706    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused"
Oct 30 19:09:12.276339 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:12.269549    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused" start-of-body=
Oct 30 19:09:12.276554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:12.276529    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused"
Oct 30 19:09:12.996149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:12.979511    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused" start-of-body=
Oct 30 19:09:12.996149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:12.979567    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" podUID="b330c3ad-b62a-4146-9626-d8e9045512ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused"
Oct 30 19:09:13.273961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:13.272393    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused" start-of-body=
Oct 30 19:09:13.273961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:13.272475    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused"
Oct 30 19:09:13.273961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:13.273269    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused" start-of-body=
Oct 30 19:09:13.273961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:13.273311    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused"
Oct 30 19:09:13.552192 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:09:13.549914    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:09:13.943121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:13.924257    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:14.004123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:13.999298    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused" start-of-body=
Oct 30 19:09:14.004123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:13.999362    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" podUID="b330c3ad-b62a-4146-9626-d8e9045512ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused"
Oct 30 19:09:14.265273 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:14.264516    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused" start-of-body=
Oct 30 19:09:14.265273 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:14.264597    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused"
Oct 30 19:09:14.285191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:14.283425    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused" start-of-body=
Oct 30 19:09:14.285191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:14.283494    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused"
Oct 30 19:09:14.957467 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:14.957429    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused" start-of-body=
Oct 30 19:09:14.957773 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:14.957743    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" podUID="b330c3ad-b62a-4146-9626-d8e9045512ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused"
Oct 30 19:09:15.268179 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:15.260566    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused" start-of-body=
Oct 30 19:09:15.268179 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:15.260700    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.226:15021/healthz/ready\": dial tcp 10.132.0.226:15021: connect: connection refused"
Oct 30 19:09:15.268179 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:15.267517    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused" start-of-body=
Oct 30 19:09:15.268179 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:15.267562    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.227:15021/healthz/ready\": dial tcp 10.132.0.227:15021: connect: connection refused"
Oct 30 19:09:15.369261 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:15.369224    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:15.962443 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:15.962403    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused" start-of-body=
Oct 30 19:09:15.962718 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:15.962688    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" podUID="b330c3ad-b62a-4146-9626-d8e9045512ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused"
Oct 30 19:09:16.292451 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:16.292369    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:16.294631 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:16.294593    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:16.295079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:16.295055    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:16.295757 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:16.295735    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:16.296278 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:16.296262    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:09:16.296725 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:16.296709    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:09:16.544317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:16.544058    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:16.989999 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:16.989961    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused" start-of-body=
Oct 30 19:09:16.990233 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:16.990037    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" podUID="b330c3ad-b62a-4146-9626-d8e9045512ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.228:15021/healthz/ready\": dial tcp 10.132.0.228:15021: connect: connection refused"
Oct 30 19:09:17.231687 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:17.231655    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:17.964142 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:17.963545    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:17.964679 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:17.964263    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:17.965032 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:17.965010    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:09:18.086955 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:18.086918    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:18.809378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:18.809342    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:19.612735 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:19.612698    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:20.729122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:20.724557    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:21.710156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:21.710046    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:22.664673 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:22.664625    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:23.889835 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:23.889795    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:24.741531 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:24.741478    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:25.557899 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:25.557860    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:26.476952 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:26.476916    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:27.380708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:27.379970    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:28.436849 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:28.436811    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:29.097265 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:29.095086    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:29.344269 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:29.344224    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.217:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:09:29.344460 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:29.344282    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.217:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:09:29.866798 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:29.866702    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:30.682863 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:30.681795    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"]
Oct 30 19:09:30.807090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:30.806872    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:30.989970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:30.989889    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:31.771405 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:31.771307    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:32.564979 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:32.564939    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:33.502222 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:33.501617    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:33.665781 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:33.665744    4925 generic.go:356] "Generic (PLEG): container finished" podID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerID="c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634" exitCode=0
Oct 30 19:09:33.666305 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:33.665823    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" event={"ID":"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8","Type":"ContainerDied","Data":"c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634"}
Oct 30 19:09:33.666446 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:33.666424    4925 scope.go:122] "RemoveContainer" containerID="c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634"
Oct 30 19:09:33.668736 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:33.668702    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:33.668883 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:33.668794    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:33.668883 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:33.668839    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:34.292452 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:34.281787    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:34.700788 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:34.700740    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" event={"ID":"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8","Type":"ContainerStarted","Data":"27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03"}
Oct 30 19:09:34.805119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:34.805065    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"]
Oct 30 19:09:34.807217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:34.805343    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf" podUID="7ddb6fef-cf25-4773-b561-eea818ebc218" containerName="maas-api" containerID="cri-o://36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f" gracePeriod=30
Oct 30 19:09:34.951233 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:34.951149    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:35.313224 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.288054    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:35.423632 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.401719    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:09:35.423632 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.412813    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.217:15021/healthz/ready\": dial tcp 10.132.0.217:15021: connect: connection refused" start-of-body=
Oct 30 19:09:35.423632 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.412879    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.217:15021/healthz/ready\": dial tcp 10.132.0.217:15021: connect: connection refused"
Oct 30 19:09:35.547257 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.547224    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:35.634911 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.628302    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:09:35.710123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.709028    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j" podUID="238cc3c0-8010-4e26-8eaa-927d1cdf823d" containerName="payload-processing" containerID="cri-o://b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0" gracePeriod=25
Oct 30 19:09:35.756087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.750556    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-maas-api-tls\") pod \"7ddb6fef-cf25-4773-b561-eea818ebc218\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") "
Oct 30 19:09:35.784518 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.766352    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-metrics-certs\") pod \"7ddb6fef-cf25-4773-b561-eea818ebc218\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") "
Oct 30 19:09:35.784518 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.766634    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ddb6fef-cf25-4773-b561-eea818ebc218-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ddb6fef-cf25-4773-b561-eea818ebc218-openshift-service-ca\") pod \"7ddb6fef-cf25-4773-b561-eea818ebc218\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") "
Oct 30 19:09:35.784518 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.767134    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ddb6fef-cf25-4773-b561-eea818ebc218-kube-api-access-zvhh7\" (UniqueName: \"kubernetes.io/projected/7ddb6fef-cf25-4773-b561-eea818ebc218-kube-api-access-zvhh7\") pod \"7ddb6fef-cf25-4773-b561-eea818ebc218\" (UID: \"7ddb6fef-cf25-4773-b561-eea818ebc218\") "
Oct 30 19:09:35.922710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.918807    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-maas-api-tls" pod "7ddb6fef-cf25-4773-b561-eea818ebc218" (UID: "7ddb6fef-cf25-4773-b561-eea818ebc218"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:09:35.922710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.919470    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ddb6fef-cf25-4773-b561-eea818ebc218-kube-api-access-zvhh7" pod "7ddb6fef-cf25-4773-b561-eea818ebc218" (UID: "7ddb6fef-cf25-4773-b561-eea818ebc218"). InnerVolumeSpecName "kube-api-access-zvhh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:35.922710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.921656    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-metrics-certs" pod "7ddb6fef-cf25-4773-b561-eea818ebc218" (UID: "7ddb6fef-cf25-4773-b561-eea818ebc218"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:09:35.922710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.921821    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ddb6fef-cf25-4773-b561-eea818ebc218-openshift-service-ca" pod "7ddb6fef-cf25-4773-b561-eea818ebc218" (UID: "7ddb6fef-cf25-4773-b561-eea818ebc218"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:35.926695 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.923848    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:35.936671 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.936630    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"]
Oct 30 19:09:35.940882 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.940324    4925 cpu_manager.go:414] "containerMap: removing container" podUID="7ddb6fef-cf25-4773-b561-eea818ebc218" containerName="maas-api"
Oct 30 19:09:35.940882 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.940348    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ddb6fef-cf25-4773-b561-eea818ebc218" containerName="maas-api"
Oct 30 19:09:35.941062 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.940887    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ddb6fef-cf25-4773-b561-eea818ebc218" containerName="maas-api"
Oct 30 19:09:35.951596 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.951566    4925 generic.go:356] "Generic (PLEG): container finished" podID="7ddb6fef-cf25-4773-b561-eea818ebc218" containerID="36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f" exitCode=0
Oct 30 19:09:35.951798 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.951777    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"
Oct 30 19:09:35.954662 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.954416    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"]
Oct 30 19:09:35.954662 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.954458    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf" event={"ID":"7ddb6fef-cf25-4773-b561-eea818ebc218","Type":"ContainerDied","Data":"36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f"}
Oct 30 19:09:35.954662 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.954491    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf" event={"ID":"7ddb6fef-cf25-4773-b561-eea818ebc218","Type":"ContainerDied","Data":"2651a5fcdc815ce5a955ddcf6bd0539f10b85f5286a53e896c0562449c9d2a22"}
Oct 30 19:09:35.954662 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.954512    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:35.954662 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.954520    4925 scope.go:122] "RemoveContainer" containerID="36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f"
Oct 30 19:09:35.997917 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:35.997889    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-c164d6-openshift-default-dockercfg-c42sr\""
Oct 30 19:09:36.013885 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.013562    4925 scope.go:122] "RemoveContainer" containerID="36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f"
Oct 30 19:09:36.014226 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:09:36.014118    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f\": container with ID starting with 36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f not found: ID does not exist" containerID="36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f"
Oct 30 19:09:36.014226 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.014161    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f"} err="failed to get container status \"36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f\": rpc error: code = NotFound desc = could not find container \"36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f\": container with ID starting with 36b2d14a9a1b04225e36fae10d5ee5791ad2967d62e10b8c7b0e594a5fc0ee6f not found: ID does not exist"
Oct 30 19:09:36.019961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.019873    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:36.019961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.019895    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ddb6fef-cf25-4773-b561-eea818ebc218-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:36.019961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.019906    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvhh7\" (UniqueName: \"kubernetes.io/projected/7ddb6fef-cf25-4773-b561-eea818ebc218-kube-api-access-zvhh7\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:36.019961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.019915    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ddb6fef-cf25-4773-b561-eea818ebc218-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:36.063180 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.063148    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"]
Oct 30 19:09:36.110992 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.110961    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-c164d6-5456d697d7-5tsgf"]
Oct 30 19:09:36.120846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.120811    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istiod-ca-cert\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.120977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.120870    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-credential-socket\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.121140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.120982    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-data\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.121140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.121037    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-certs\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.121140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.121072    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-envoy\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.121365 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.121278    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-socket\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.121365 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.121327    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9ckh\" (UniqueName: \"kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-kube-api-access-s9ckh\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.121509 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.121479    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-token\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.121569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.121555    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-podinfo\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.204482 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.204353    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.223662    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-token\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.223793    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-podinfo\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.223899    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istiod-ca-cert\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.223953    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-credential-socket\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.224039    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-data\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.224112    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-certs\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.224147    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-envoy\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.225032    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istiod-ca-cert\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.225924    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-certs\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.226214    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-credential-socket\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.226441    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-data\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.226893    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-socket\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.226948    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9ckh\" (UniqueName: \"kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-kube-api-access-s9ckh\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.227621    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-socket\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.232743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.232555    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-envoy\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.233667 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.233618    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-podinfo\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.241334 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.241308    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9ckh\" (UniqueName: \"kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-kube-api-access-s9ckh\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.241819 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.241795    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-token\") pod \"e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") " pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.320932 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.320902    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:36.323436 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.323412    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"
Oct 30 19:09:36.405120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.404339    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.217:15021/healthz/ready\": dial tcp 10.132.0.217:15021: connect: connection refused" start-of-body=
Oct 30 19:09:36.405120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.404410    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.217:15021/healthz/ready\": dial tcp 10.132.0.217:15021: connect: connection refused"
Oct 30 19:09:36.442228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.442194    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/238cc3c0-8010-4e26-8eaa-927d1cdf823d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/238cc3c0-8010-4e26-8eaa-927d1cdf823d-plugins-config-volume\") pod \"238cc3c0-8010-4e26-8eaa-927d1cdf823d\" (UID: \"238cc3c0-8010-4e26-8eaa-927d1cdf823d\") "
Oct 30 19:09:36.442642 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.442619    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/238cc3c0-8010-4e26-8eaa-927d1cdf823d-kube-api-access-vfgkz\" (UniqueName: \"kubernetes.io/projected/238cc3c0-8010-4e26-8eaa-927d1cdf823d-kube-api-access-vfgkz\") pod \"238cc3c0-8010-4e26-8eaa-927d1cdf823d\" (UID: \"238cc3c0-8010-4e26-8eaa-927d1cdf823d\") "
Oct 30 19:09:36.450496 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.450462    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/238cc3c0-8010-4e26-8eaa-927d1cdf823d-plugins-config-volume" pod "238cc3c0-8010-4e26-8eaa-927d1cdf823d" (UID: "238cc3c0-8010-4e26-8eaa-927d1cdf823d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:36.450895 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.450868    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/238cc3c0-8010-4e26-8eaa-927d1cdf823d-kube-api-access-vfgkz" pod "238cc3c0-8010-4e26-8eaa-927d1cdf823d" (UID: "238cc3c0-8010-4e26-8eaa-927d1cdf823d"). InnerVolumeSpecName "kube-api-access-vfgkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:36.487049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.486969    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ddb6fef-cf25-4773-b561-eea818ebc218" path="/var/lib/kubelet/pods/7ddb6fef-cf25-4773-b561-eea818ebc218/volumes"
Oct 30 19:09:36.550108 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.550071    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/238cc3c0-8010-4e26-8eaa-927d1cdf823d-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:36.550241 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.550119    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfgkz\" (UniqueName: \"kubernetes.io/projected/238cc3c0-8010-4e26-8eaa-927d1cdf823d-kube-api-access-vfgkz\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:36.750874 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.750841    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"]
Oct 30 19:09:36.770114 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.756646    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:36.770114 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.756737    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:36.770114 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:36.756778    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:09:37.032140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.031661    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:37.093127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.092245    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm" event={"ID":"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1","Type":"ContainerStarted","Data":"97059fa806eacedde92937361eb838e43da4cb62a4ec2f132847dab6436319c0"}
Oct 30 19:09:37.095380 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.095354    4925 generic.go:356] "Generic (PLEG): container finished" podID="238cc3c0-8010-4e26-8eaa-927d1cdf823d" containerID="b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0" exitCode=0
Oct 30 19:09:37.095572 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.095557    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j" event={"ID":"238cc3c0-8010-4e26-8eaa-927d1cdf823d","Type":"ContainerDied","Data":"b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0"}
Oct 30 19:09:37.095634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.095596    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j" event={"ID":"238cc3c0-8010-4e26-8eaa-927d1cdf823d","Type":"ContainerDied","Data":"efb4f28a9ea1f22c9659d2c44808d8627f1cf31b9d837d8ed1c9bf01f2bf6761"}
Oct 30 19:09:37.095634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.095613    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"
Oct 30 19:09:37.095993 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.095616    4925 scope.go:122] "RemoveContainer" containerID="b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0"
Oct 30 19:09:37.118065 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.118041    4925 scope.go:122] "RemoveContainer" containerID="b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0"
Oct 30 19:09:37.118415 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:09:37.118393    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0\": container with ID starting with b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0 not found: ID does not exist" containerID="b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0"
Oct 30 19:09:37.118523 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.118429    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0"} err="failed to get container status \"b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0\": rpc error: code = NotFound desc = could not find container \"b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0\": container with ID starting with b53a4fad2b18d29b1ea51e9806c01d5c2c0313980d3cb559de209350d77bf7c0 not found: ID does not exist"
Oct 30 19:09:37.144022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.143953    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm" podStartSLOduration=9.143935154 podStartE2EDuration="9.143935154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:37.139884581 +0000 UTC m=+1998.856229662" watchObservedRunningTime="2026-10-30 19:09:37.143935154 +0000 UTC m=+1998.860280232"
Oct 30 19:09:37.201149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.201085    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"]
Oct 30 19:09:37.208511 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.208486    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j"]
Oct 30 19:09:37.232714 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.232673    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-autores-c164d6-7b8dc947d8-fmg5j" podUID="238cc3c0-8010-4e26-8eaa-927d1cdf823d" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.216:50052\" within 1s: context deadline exceeded"
Oct 30 19:09:37.232843 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:09:37.232737    4925 logging.go:55] [core] [Channel #5539 SubChannel #5540]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.216:50052", ServerName: "10.132.0.216:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.216:50052: operation was canceled"
Oct 30 19:09:37.321347 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.321269    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:37.403126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.402654    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.217:15021/healthz/ready\": dial tcp 10.132.0.217:15021: connect: connection refused" start-of-body=
Oct 30 19:09:37.403126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:37.402716    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.217:15021/healthz/ready\": dial tcp 10.132.0.217:15021: connect: connection refused"
Oct 30 19:09:38.063341 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:38.063305    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:38.210167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:38.208629    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm" event={"ID":"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1","Type":"ContainerStarted","Data":"f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7"}
Oct 30 19:09:38.323580 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:38.323499    4925 patch_prober.go:29] interesting pod/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.229:15021/healthz/ready\": dial tcp 10.132.0.229:15021: connect: connection refused" start-of-body=
Oct 30 19:09:38.323580 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:38.323559    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm" podUID="7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.229:15021/healthz/ready\": dial tcp 10.132.0.229:15021: connect: connection refused"
Oct 30 19:09:38.394625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:38.394588    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.217:15021/healthz/ready\": dial tcp 10.132.0.217:15021: connect: connection refused" start-of-body=
Oct 30 19:09:38.394920 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:38.394891    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.217:15021/healthz/ready\": dial tcp 10.132.0.217:15021: connect: connection refused"
Oct 30 19:09:38.440983 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:38.440956    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="238cc3c0-8010-4e26-8eaa-927d1cdf823d" path="/var/lib/kubelet/pods/238cc3c0-8010-4e26-8eaa-927d1cdf823d/volumes"
Oct 30 19:09:39.280006 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:39.279975    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:39.323564 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:39.323527    4925 patch_prober.go:29] interesting pod/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.229:15021/healthz/ready\": dial tcp 10.132.0.229:15021: connect: connection refused" start-of-body=
Oct 30 19:09:39.323867 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:39.323831    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm" podUID="7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.229:15021/healthz/ready\": dial tcp 10.132.0.229:15021: connect: connection refused"
Oct 30 19:09:39.399279 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:39.399243    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.217:15021/healthz/ready\": dial tcp 10.132.0.217:15021: connect: connection refused" start-of-body=
Oct 30 19:09:39.399551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:39.399523    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.217:15021/healthz/ready\": dial tcp 10.132.0.217:15021: connect: connection refused"
Oct 30 19:09:40.323807 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:40.323765    4925 patch_prober.go:29] interesting pod/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.229:15021/healthz/ready\": dial tcp 10.132.0.229:15021: connect: connection refused" start-of-body=
Oct 30 19:09:40.324445 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:40.324394    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm" podUID="7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.229:15021/healthz/ready\": dial tcp 10.132.0.229:15021: connect: connection refused"
Oct 30 19:09:40.395459 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:40.395420    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.217:15021/healthz/ready\": dial tcp 10.132.0.217:15021: connect: connection refused" start-of-body=
Oct 30 19:09:40.395722 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:40.395696    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.217:15021/healthz/ready\": dial tcp 10.132.0.217:15021: connect: connection refused"
Oct 30 19:09:40.686120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:40.677633    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:41.321885 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:41.321846    4925 patch_prober.go:29] interesting pod/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.229:15021/healthz/ready\": dial tcp 10.132.0.229:15021: connect: connection refused" start-of-body=
Oct 30 19:09:41.322071 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:41.321911    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm" podUID="7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.229:15021/healthz/ready\": dial tcp 10.132.0.229:15021: connect: connection refused"
Oct 30 19:09:41.397550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:41.397521    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:09:41.400152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:41.398186    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:09:41.400152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:41.398965    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:09:41.706369 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:41.697258    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:42.325242 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:42.325202    4925 patch_prober.go:29] interesting pod/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.229:15021/healthz/ready\": dial tcp 10.132.0.229:15021: connect: connection refused" start-of-body=
Oct 30 19:09:42.325585 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:42.325546    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm" podUID="7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.229:15021/healthz/ready\": dial tcp 10.132.0.229:15021: connect: connection refused"
Oct 30 19:09:42.643729 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:42.643613    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:43.320131 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:09:43.317781    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:09:43.322300 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:43.322270    4925 patch_prober.go:29] interesting pod/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.229:15021/healthz/ready\": dial tcp 10.132.0.229:15021: connect: connection refused" start-of-body=
Oct 30 19:09:43.322429 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:43.322330    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm" podUID="7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.229:15021/healthz/ready\": dial tcp 10.132.0.229:15021: connect: connection refused"
Oct 30 19:09:43.523122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:43.513741    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:43.780141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:43.777660    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:43.887185 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:43.887147    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"]
Oct 30 19:09:43.887632 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:43.887597    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm" podUID="7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" containerName="istio-proxy" containerID="cri-o://f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7" gracePeriod=30
Oct 30 19:09:44.033334 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:44.029824    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:44.704121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:44.703798    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:45.435164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:45.434462    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:45.833661 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:45.833571    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:46.351285 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:46.351239    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:46.578122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:46.577616    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:47.298784 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:47.298193    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:48.114157 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:48.108338    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:48.792592 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:48.792558    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:49.516565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.516528    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:49.813660 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.813632    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:49.924234 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924174    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-socket\") pod \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") "
Oct 30 19:09:49.924425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924273    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-credential-socket\") pod \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") "
Oct 30 19:09:49.924425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924332    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-podinfo\") pod \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") "
Oct 30 19:09:49.924425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924381    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-data\") pod \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") "
Oct 30 19:09:49.924634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924424    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-kube-api-access-s9ckh\" (UniqueName: \"kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-kube-api-access-s9ckh\") pod \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") "
Oct 30 19:09:49.924634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924435    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-socket" pod "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" (UID: "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:49.924634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924533    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istiod-ca-cert\") pod \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") "
Oct 30 19:09:49.924634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924568    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-credential-socket" pod "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" (UID: "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:49.924864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924694    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-data" pod "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" (UID: "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:49.924864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924734    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-token\" (UniqueName: \"kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-token\") pod \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") "
Oct 30 19:09:49.924864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924827    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-envoy\") pod \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") "
Oct 30 19:09:49.925053 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924867    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-certs\") pod \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\" (UID: \"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1\") "
Oct 30 19:09:49.925053 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.924894    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istiod-ca-cert" pod "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" (UID: "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:49.925221 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.925191    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-certs" pod "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" (UID: "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:49.926304 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.926276    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:49.926304 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.926306    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:49.926513 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.926321    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:49.926513 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.926334    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:49.926513 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.926347    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:49.927112 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.927068    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-podinfo" pod "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" (UID: "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:09:49.927371 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.927344    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-token" pod "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" (UID: "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:49.927479 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.927405    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-envoy" pod "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" (UID: "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:49.927815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:49.927795    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-kube-api-access-s9ckh" pod "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" (UID: "7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1"). InnerVolumeSpecName "kube-api-access-s9ckh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:50.028037 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.027953    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:50.028037 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.027991    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:50.028037 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.028008    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:50.028037 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.028023    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9ckh\" (UniqueName: \"kubernetes.io/projected/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1-kube-api-access-s9ckh\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:09:50.166173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.166060    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:50.591490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.591450    4925 generic.go:356] "Generic (PLEG): container finished" podID="7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" containerID="f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7" exitCode=0
Oct 30 19:09:50.591973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.591511    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm" event={"ID":"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1","Type":"ContainerDied","Data":"f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7"}
Oct 30 19:09:50.591973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.591564    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm" event={"ID":"7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1","Type":"ContainerDied","Data":"97059fa806eacedde92937361eb838e43da4cb62a4ec2f132847dab6436319c0"}
Oct 30 19:09:50.591973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.591597    4925 scope.go:122] "RemoveContainer" containerID="f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7"
Oct 30 19:09:50.591973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.591783    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"
Oct 30 19:09:50.608217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.608192    4925 scope.go:122] "RemoveContainer" containerID="f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7"
Oct 30 19:09:50.608522 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:09:50.608484    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7\": container with ID starting with f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7 not found: ID does not exist" containerID="f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7"
Oct 30 19:09:50.608613 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.608529    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7"} err="failed to get container status \"f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7\": rpc error: code = NotFound desc = could not find container \"f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7\": container with ID starting with f42ef13f0efe396b14eb253e89af29172a9e56c7175ed1134f546a53d36102b7 not found: ID does not exist"
Oct 30 19:09:50.615670 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.615647    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"]
Oct 30 19:09:50.620558 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.620534    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-c164d6-openshift-default-74c7bf49f9-pp2sm"]
Oct 30 19:09:50.945124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:50.942592    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:51.864815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:51.864784    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:52.425165 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:52.425129    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" path="/var/lib/kubelet/pods/7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1/volumes"
Oct 30 19:09:52.565969 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:52.565937    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:52.768140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:52.768037    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:53.835701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:53.835666    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:54.098965 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.098876    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"]
Oct 30 19:09:54.099867 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.099846    4925 cpu_manager.go:414] "containerMap: removing container" podUID="7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" containerName="istio-proxy"
Oct 30 19:09:54.099867 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.099866    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" containerName="istio-proxy"
Oct 30 19:09:54.100042 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.099886    4925 cpu_manager.go:414] "containerMap: removing container" podUID="238cc3c0-8010-4e26-8eaa-927d1cdf823d" containerName="payload-processing"
Oct 30 19:09:54.100042 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.099891    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="238cc3c0-8010-4e26-8eaa-927d1cdf823d" containerName="payload-processing"
Oct 30 19:09:54.100042 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.100029    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f32f6cc-3ddb-4d48-b32c-62c6adff8fc1" containerName="istio-proxy"
Oct 30 19:09:54.100042 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.100040    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="238cc3c0-8010-4e26-8eaa-927d1cdf823d" containerName="payload-processing"
Oct 30 19:09:54.105755 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.105731    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"
Oct 30 19:09:54.109817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.109785    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"]
Oct 30 19:09:54.110471 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.110441    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-37f2b8\""
Oct 30 19:09:54.116648 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.116627    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"]
Oct 30 19:09:54.116905 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.116746    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"
Oct 30 19:09:54.120443 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.120399    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:54.139986 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.139382    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"]
Oct 30 19:09:54.146134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.142534    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-37f2b8-dockercfg-gztpl\""
Oct 30 19:09:54.287694 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.287567    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be5cb88e-59da-4585-9114-484c0773dc19-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f\" (UID: \"be5cb88e-59da-4585-9114-484c0773dc19\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"
Oct 30 19:09:54.305329 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.305275    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhqmh\" (UniqueName: \"kubernetes.io/projected/be5cb88e-59da-4585-9114-484c0773dc19-kube-api-access-zhqmh\") pod \"payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f\" (UID: \"be5cb88e-59da-4585-9114-484c0773dc19\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"
Oct 30 19:09:54.341274 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.307389    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgpxl\" (UniqueName: \"kubernetes.io/projected/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-kube-api-access-qgpxl\") pod \"payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7\" (UID: \"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"
Oct 30 19:09:54.341274 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.307631    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7\" (UID: \"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"
Oct 30 19:09:54.473122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.465138    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhqmh\" (UniqueName: \"kubernetes.io/projected/be5cb88e-59da-4585-9114-484c0773dc19-kube-api-access-zhqmh\") pod \"payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f\" (UID: \"be5cb88e-59da-4585-9114-484c0773dc19\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"
Oct 30 19:09:54.473122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.472259    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgpxl\" (UniqueName: \"kubernetes.io/projected/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-kube-api-access-qgpxl\") pod \"payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7\" (UID: \"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"
Oct 30 19:09:54.499123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.497939    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7\" (UID: \"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"
Oct 30 19:09:54.499123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.498083    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be5cb88e-59da-4585-9114-484c0773dc19-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f\" (UID: \"be5cb88e-59da-4585-9114-484c0773dc19\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"
Oct 30 19:09:54.499397 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.499337    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be5cb88e-59da-4585-9114-484c0773dc19-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f\" (UID: \"be5cb88e-59da-4585-9114-484c0773dc19\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"
Oct 30 19:09:54.532116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.500327    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7\" (UID: \"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"
Oct 30 19:09:54.532375 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.530510    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:54.599118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.598305    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhqmh\" (UniqueName: \"kubernetes.io/projected/be5cb88e-59da-4585-9114-484c0773dc19-kube-api-access-zhqmh\") pod \"payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f\" (UID: \"be5cb88e-59da-4585-9114-484c0773dc19\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"
Oct 30 19:09:54.599118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.598743    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgpxl\" (UniqueName: \"kubernetes.io/projected/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-kube-api-access-qgpxl\") pod \"payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7\" (UID: \"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"
Oct 30 19:09:54.756021 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.755939    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"
Oct 30 19:09:54.790523 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:54.790490    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"
Oct 30 19:09:55.106008 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:55.104300    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:55.163277 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:55.162549    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"]
Oct 30 19:09:55.166971 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:55.166928    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"]
Oct 30 19:09:55.653310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:55.653261    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7" event={"ID":"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae","Type":"ContainerStarted","Data":"10cca33001b0028e19009cd7cea77fceb3792b6db6d6042004f14a6bc2892721"}
Oct 30 19:09:55.653466 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:55.653330    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7" event={"ID":"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae","Type":"ContainerStarted","Data":"0fc23c27b0c93effe72b3d9701d55de07a2afe43f1b0ac9262f51e85ce337f88"}
Oct 30 19:09:55.653466 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:55.653352    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"
Oct 30 19:09:55.655125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:55.655084    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f" event={"ID":"be5cb88e-59da-4585-9114-484c0773dc19","Type":"ContainerStarted","Data":"cdc8d730ff1a66d2db09bd79f45897d1f4d57f789af184aa7ac45bf880c94abe"}
Oct 30 19:09:55.655277 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:55.655144    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f" event={"ID":"be5cb88e-59da-4585-9114-484c0773dc19","Type":"ContainerStarted","Data":"48705cbdcd185dbc28021f18a27dab40f1ff328f46ec7ea01a334ab6ca957274"}
Oct 30 19:09:55.655277 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:55.655182    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"
Oct 30 19:09:55.678494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:55.673623    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7" podStartSLOduration=1.673606148 podStartE2EDuration="1.673606148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:55.671153077 +0000 UTC m=+2017.387498152" watchObservedRunningTime="2026-10-30 19:09:55.673606148 +0000 UTC m=+2017.389951225"
Oct 30 19:09:55.689888 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:55.689832    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f" podStartSLOduration=1.68981497 podStartE2EDuration="1.68981497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:55.688117123 +0000 UTC m=+2017.404462197" watchObservedRunningTime="2026-10-30 19:09:55.68981497 +0000 UTC m=+2017.406160048"
Oct 30 19:09:55.948122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:55.946035    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:56.610123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:56.608265    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:57.380782 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:57.380557    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:58.094118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:58.093593    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:58.729756 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:58.729401    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:59.418134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:09:59.399371    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:00.088191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.084957    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:00.728458 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.728427    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"]
Oct 30 19:10:00.733995 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.733965    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:00.742953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.742922    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-e67d95-openshift-default-dockercfg-jdrgt\""
Oct 30 19:10:00.750629 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.750603    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"]
Oct 30 19:10:00.909119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.907169    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:00.909398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.909369    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-certs\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:00.909501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.909413    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f531c729-0f21-45d5-b635-e8613eda326b-istiod-ca-cert\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:00.909501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.909484    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f531c729-0f21-45d5-b635-e8613eda326b-istio-podinfo\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:00.909643 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.909561    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-istio-token\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:00.909706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.909651    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-credential-socket\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:00.909706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.909681    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-socket\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:00.909832 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.909708    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-envoy\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:00.909832 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.909741    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqpk6\" (UniqueName: \"kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-kube-api-access-vqpk6\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:00.910027 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:00.909836    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-data\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.011975 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.011887    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-data\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.012158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.012134    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-certs\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.012236 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.012158    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f531c729-0f21-45d5-b635-e8613eda326b-istiod-ca-cert\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.012236 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.012221    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f531c729-0f21-45d5-b635-e8613eda326b-istio-podinfo\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.012362 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.012303    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-istio-token\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.012362 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.012336    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-data\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.012518 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.012498    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-certs\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.012605 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.012572    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-credential-socket\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.012697 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.012602    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-socket\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.012697 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.012639    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-envoy\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.012697 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.012671    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqpk6\" (UniqueName: \"kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-kube-api-access-vqpk6\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.012954 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.012934    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f531c729-0f21-45d5-b635-e8613eda326b-istiod-ca-cert\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.013035 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.012933    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-credential-socket\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.013035 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.013009    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-socket\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.015412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.015385    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-envoy\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.015705 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.015683    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f531c729-0f21-45d5-b635-e8613eda326b-istio-podinfo\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.022657 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.022633    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqpk6\" (UniqueName: \"kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-kube-api-access-vqpk6\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.022940 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.022922    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-istio-token\") pod \"e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") " pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.053968 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.053945    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:01.586123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.561898    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"]
Oct 30 19:10:01.600947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.600897    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:10:01.601161 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.601005    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:10:01.601161 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.601055    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:10:01.814068 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:01.813965    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb" event={"ID":"f531c729-0f21-45d5-b635-e8613eda326b","Type":"ContainerStarted","Data":"ada1c36bb29c22cdbf750ca442e9ed90a82529001ef67aa4962bad21e4817a52"}
Oct 30 19:10:02.089208 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:02.085782    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:02.824153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:02.824089    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb" event={"ID":"f531c729-0f21-45d5-b635-e8613eda326b","Type":"ContainerStarted","Data":"aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44"}
Oct 30 19:10:02.854546 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:02.854486    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb" podStartSLOduration=2.854473199 podStartE2EDuration="2.854473199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:02.848206889 +0000 UTC m=+2024.564551966" watchObservedRunningTime="2026-10-30 19:10:02.854473199 +0000 UTC m=+2024.570818274"
Oct 30 19:10:03.034828 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:03.034774    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:03.064534 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:03.064502    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:03.064907 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:03.064825    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.232:15021/healthz/ready\": dial tcp 10.132.0.232:15021: connect: connection refused" start-of-body=
Oct 30 19:10:03.065079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:03.065057    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb" podUID="f531c729-0f21-45d5-b635-e8613eda326b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.232:15021/healthz/ready\": dial tcp 10.132.0.232:15021: connect: connection refused"
Oct 30 19:10:03.693143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:03.693083    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:03.988179 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:03.987161    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:04.054558 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:04.054514    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.232:15021/healthz/ready\": dial tcp 10.132.0.232:15021: connect: connection refused" start-of-body=
Oct 30 19:10:04.055408 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:04.054859    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb" podUID="f531c729-0f21-45d5-b635-e8613eda326b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.232:15021/healthz/ready\": dial tcp 10.132.0.232:15021: connect: connection refused"
Oct 30 19:10:04.277137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:04.277027    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:04.544035 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:04.541202    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:04.772191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:04.765252    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:05.069566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:05.069523    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.232:15021/healthz/ready\": dial tcp 10.132.0.232:15021: connect: connection refused" start-of-body=
Oct 30 19:10:05.070166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:05.070133    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb" podUID="f531c729-0f21-45d5-b635-e8613eda326b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.232:15021/healthz/ready\": dial tcp 10.132.0.232:15021: connect: connection refused"
Oct 30 19:10:05.270965 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:05.270931    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:05.499005 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:05.498967    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:05.819356 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:05.819249    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:06.019566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:06.019534    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:06.054927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:06.054894    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.232:15021/healthz/ready\": dial tcp 10.132.0.232:15021: connect: connection refused" start-of-body=
Oct 30 19:10:06.055088 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:06.054952    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb" podUID="f531c729-0f21-45d5-b635-e8613eda326b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.232:15021/healthz/ready\": dial tcp 10.132.0.232:15021: connect: connection refused"
Oct 30 19:10:06.332569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:06.280991    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:06.678251 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:06.678221    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"
Oct 30 19:10:06.680180 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:06.680160    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"
Oct 30 19:10:06.903186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:06.890624    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:07.054848 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:07.054743    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.232:15021/healthz/ready\": dial tcp 10.132.0.232:15021: connect: connection refused" start-of-body=
Oct 30 19:10:07.054848 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:07.054832    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb" podUID="f531c729-0f21-45d5-b635-e8613eda326b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.232:15021/healthz/ready\": dial tcp 10.132.0.232:15021: connect: connection refused"
Oct 30 19:10:07.779738 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:07.779707    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:08.055135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:08.055019    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.232:15021/healthz/ready\": dial tcp 10.132.0.232:15021: connect: connection refused" start-of-body=
Oct 30 19:10:08.055301 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:08.055091    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb" podUID="f531c729-0f21-45d5-b635-e8613eda326b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.232:15021/healthz/ready\": dial tcp 10.132.0.232:15021: connect: connection refused"
Oct 30 19:10:08.590786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:08.590749    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:09.060651 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:09.060610    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:09.061204 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:09.061182    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:09.061957 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:09.061932    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:10:09.335060 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:09.334979    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:10.032333 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:10.030712    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:10.782021 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:10.781986    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:11.033914 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:11.033833    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:11.568206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:11.568169    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:12.506716 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:12.506678    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:13.299728 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:13.299689    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:13.335492 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:10:13.335449    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:10:13.998444 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:13.998408    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:14.201839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:14.201806    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:14.405275 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:14.405119    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:14.989642 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:14.987440    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:15.639223 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:15.639110    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:16.501684 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:16.501652    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:17.207610 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:17.207562    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:17.873130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:17.871438    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:18.573916 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:18.573880    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:18.646629 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:18.646597    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"]
Oct 30 19:10:19.195304 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:19.194822    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:19.663929 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:19.663888    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"]
Oct 30 19:10:19.882617 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:19.882583    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:20.507151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:20.507078    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:21.136987 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:21.136946    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:21.774657 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:21.774577    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:22.622215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:22.616385    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:23.169309 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:23.169277    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:23.648238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:23.648154    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb" podUID="29a53a63-606a-4d0a-a548-b5f66cc2e988" containerName="payload-processing" containerID="cri-o://c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff" gracePeriod=25
Oct 30 19:10:23.808145 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:23.799642    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:24.030649 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.030625    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:10:24.119374 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.119339    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-routing-overlay-volume\") pod \"29a53a63-606a-4d0a-a548-b5f66cc2e988\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") "
Oct 30 19:10:24.119573 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.119447    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-provider-credentials\" (UniqueName: \"kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-provider-credentials\") pod \"29a53a63-606a-4d0a-a548-b5f66cc2e988\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") "
Oct 30 19:10:24.119573 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.119491    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-plugins-config-volume\") pod \"29a53a63-606a-4d0a-a548-b5f66cc2e988\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") "
Oct 30 19:10:24.119573 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.119519    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-kube-api-access-btqf2\" (UniqueName: \"kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-kube-api-access-btqf2\") pod \"29a53a63-606a-4d0a-a548-b5f66cc2e988\" (UID: \"29a53a63-606a-4d0a-a548-b5f66cc2e988\") "
Oct 30 19:10:24.119825 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.119799    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-routing-overlay-volume" pod "29a53a63-606a-4d0a-a548-b5f66cc2e988" (UID: "29a53a63-606a-4d0a-a548-b5f66cc2e988"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:10:24.119968 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.119939    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-plugins-config-volume" pod "29a53a63-606a-4d0a-a548-b5f66cc2e988" (UID: "29a53a63-606a-4d0a-a548-b5f66cc2e988"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:10:24.120786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.120757    4925 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-routing-overlay-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:24.120950 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.120788    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/29a53a63-606a-4d0a-a548-b5f66cc2e988-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:24.122386 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.122359    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-kube-api-access-btqf2" pod "29a53a63-606a-4d0a-a548-b5f66cc2e988" (UID: "29a53a63-606a-4d0a-a548-b5f66cc2e988"). InnerVolumeSpecName "kube-api-access-btqf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:24.122512 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.122385    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-provider-credentials" pod "29a53a63-606a-4d0a-a548-b5f66cc2e988" (UID: "29a53a63-606a-4d0a-a548-b5f66cc2e988"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:24.222173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.222132    4925 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-provider-credentials\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:24.222173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.222167    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btqf2\" (UniqueName: \"kubernetes.io/projected/29a53a63-606a-4d0a-a548-b5f66cc2e988-kube-api-access-btqf2\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:24.291599 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.291508    4925 generic.go:356] "Generic (PLEG): container finished" podID="29a53a63-606a-4d0a-a548-b5f66cc2e988" containerID="c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff" exitCode=0
Oct 30 19:10:24.291599 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.291570    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"
Oct 30 19:10:24.291599 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.291583    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb" event={"ID":"29a53a63-606a-4d0a-a548-b5f66cc2e988","Type":"ContainerDied","Data":"c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff"}
Oct 30 19:10:24.291904 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.291659    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb" event={"ID":"29a53a63-606a-4d0a-a548-b5f66cc2e988","Type":"ContainerDied","Data":"e14643b01af2a5564dde5c47a6103bb895e91621c10dc260313840644788f52a"}
Oct 30 19:10:24.291904 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.291683    4925 scope.go:122] "RemoveContainer" containerID="c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff"
Oct 30 19:10:24.304308 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.304284    4925 scope.go:122] "RemoveContainer" containerID="c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff"
Oct 30 19:10:24.304658 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:10:24.304633    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff\": container with ID starting with c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff not found: ID does not exist" containerID="c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff"
Oct 30 19:10:24.304770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.304670    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff"} err="failed to get container status \"c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff\": rpc error: code = NotFound desc = could not find container \"c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff\": container with ID starting with c6d000454cbe86f6421fc377c692b3324e8bac934f67bbfd637c788dc9e3baff not found: ID does not exist"
Oct 30 19:10:24.323246 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.323220    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"]
Oct 30 19:10:24.327689 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.327666    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-f3d6a5/payload-processing-external-model-e2e-extmod-b-f3d6a5-f7d7p6crb"]
Oct 30 19:10:24.426343 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.426310    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29a53a63-606a-4d0a-a548-b5f66cc2e988" path="/var/lib/kubelet/pods/29a53a63-606a-4d0a-a548-b5f66cc2e988/volumes"
Oct 30 19:10:24.582019 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.581908    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:24.664684 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.664641    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2" podUID="22c9df2a-2a2d-4127-af5d-1f82c7af9726" containerName="payload-processing" containerID="cri-o://cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73" gracePeriod=25
Oct 30 19:10:24.977350 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:24.977327    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:10:25.038008 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.037978    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-provider-credentials\" (UniqueName: \"kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-provider-credentials\") pod \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") "
Oct 30 19:10:25.038218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.038053    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-kube-api-access-pzdfc\" (UniqueName: \"kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-kube-api-access-pzdfc\") pod \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") "
Oct 30 19:10:25.038218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.038148    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-plugins-config-volume\") pod \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") "
Oct 30 19:10:25.038859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.038399    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-routing-overlay-volume\") pod \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\" (UID: \"22c9df2a-2a2d-4127-af5d-1f82c7af9726\") "
Oct 30 19:10:25.038859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.038602    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-plugins-config-volume" pod "22c9df2a-2a2d-4127-af5d-1f82c7af9726" (UID: "22c9df2a-2a2d-4127-af5d-1f82c7af9726"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:10:25.038859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.038793    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-routing-overlay-volume" pod "22c9df2a-2a2d-4127-af5d-1f82c7af9726" (UID: "22c9df2a-2a2d-4127-af5d-1f82c7af9726"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:10:25.039334 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.039311    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:25.039390 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.039343    4925 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/22c9df2a-2a2d-4127-af5d-1f82c7af9726-routing-overlay-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:25.040552 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.040527    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-kube-api-access-pzdfc" pod "22c9df2a-2a2d-4127-af5d-1f82c7af9726" (UID: "22c9df2a-2a2d-4127-af5d-1f82c7af9726"). InnerVolumeSpecName "kube-api-access-pzdfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:25.040844 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.040818    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-provider-credentials" pod "22c9df2a-2a2d-4127-af5d-1f82c7af9726" (UID: "22c9df2a-2a2d-4127-af5d-1f82c7af9726"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:25.141248 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.141171    4925 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-provider-credentials\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:25.141248 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.141197    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzdfc\" (UniqueName: \"kubernetes.io/projected/22c9df2a-2a2d-4127-af5d-1f82c7af9726-kube-api-access-pzdfc\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:25.313359 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.313327    4925 generic.go:356] "Generic (PLEG): container finished" podID="22c9df2a-2a2d-4127-af5d-1f82c7af9726" containerID="cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73" exitCode=0
Oct 30 19:10:25.313559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.313384    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"
Oct 30 19:10:25.313559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.313412    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2" event={"ID":"22c9df2a-2a2d-4127-af5d-1f82c7af9726","Type":"ContainerDied","Data":"cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73"}
Oct 30 19:10:25.313559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.313477    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2" event={"ID":"22c9df2a-2a2d-4127-af5d-1f82c7af9726","Type":"ContainerDied","Data":"08d381e719ef3a69a0e0d9d1e00f130db4c358aeef1f042e91c7f1cc23d6b9e2"}
Oct 30 19:10:25.313559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.313507    4925 scope.go:122] "RemoveContainer" containerID="cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73"
Oct 30 19:10:25.329794 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.329771    4925 scope.go:122] "RemoveContainer" containerID="cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73"
Oct 30 19:10:25.330125 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:10:25.330084    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73\": container with ID starting with cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73 not found: ID does not exist" containerID="cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73"
Oct 30 19:10:25.330240 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.330139    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73"} err="failed to get container status \"cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73\": rpc error: code = NotFound desc = could not find container \"cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73\": container with ID starting with cc0ffa95beaf12f3833ce5c3aab146185fdeb9cfaf9de0e3e26d9cb5e3098c73 not found: ID does not exist"
Oct 30 19:10:25.347191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.347166    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"]
Oct 30 19:10:25.354022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.354000    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-4a477d/payload-processing-external-model-e2e-extmod-b-4a477d-6b9dg8vl2"]
Oct 30 19:10:25.368292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:25.368267    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:26.093124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:26.090883    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:26.425946 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:26.425920    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22c9df2a-2a2d-4127-af5d-1f82c7af9726" path="/var/lib/kubelet/pods/22c9df2a-2a2d-4127-af5d-1f82c7af9726/volumes"
Oct 30 19:10:26.732185 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:26.727795    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:27.469444 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:27.469401    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:28.176795 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:28.176751    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:28.805252 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:28.805185    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:29.457125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:29.456000    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:30.063124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:30.062582    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:30.848374 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:30.848180    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:31.473121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:31.471956    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:32.516131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:32.515160    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:33.182747 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:33.182704    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:33.378006 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:33.377964    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:34.157117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.154614    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:34.485546 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.485510    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:34.800088 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.799189    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"]
Oct 30 19:10:34.801486 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.801454    4925 cpu_manager.go:414] "containerMap: removing container" podUID="22c9df2a-2a2d-4127-af5d-1f82c7af9726" containerName="payload-processing"
Oct 30 19:10:34.801486 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.801484    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22c9df2a-2a2d-4127-af5d-1f82c7af9726" containerName="payload-processing"
Oct 30 19:10:34.801691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.801588    4925 cpu_manager.go:414] "containerMap: removing container" podUID="29a53a63-606a-4d0a-a548-b5f66cc2e988" containerName="payload-processing"
Oct 30 19:10:34.801691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.801597    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29a53a63-606a-4d0a-a548-b5f66cc2e988" containerName="payload-processing"
Oct 30 19:10:34.801887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.801801    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="29a53a63-606a-4d0a-a548-b5f66cc2e988" containerName="payload-processing"
Oct 30 19:10:34.801887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.801827    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="22c9df2a-2a2d-4127-af5d-1f82c7af9726" containerName="payload-processing"
Oct 30 19:10:34.816145 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.810952    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"]
Oct 30 19:10:34.816145 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.811073    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:34.816145 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.815779    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-ecc266-openshift-default-dockercfg-vsxch\""
Oct 30 19:10:34.953234 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.915809    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:34.956186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.955325    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-socket\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:34.956186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.955392    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istiod-ca-cert\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:34.956186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.955453    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-credential-socket\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:34.956186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.955528    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flh6p\" (UniqueName: \"kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-kube-api-access-flh6p\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:34.956186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.955571    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-podinfo\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:34.956186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.955637    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-envoy\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:34.956186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.955687    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-token\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:34.956186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.955791    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-data\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:34.956186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:34.955905    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-certs\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.071182 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.057309    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-credential-socket\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.116151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.073525    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-credential-socket\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.116151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.101618    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flh6p\" (UniqueName: \"kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-kube-api-access-flh6p\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.116151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.101703    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-podinfo\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.116151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.101830    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-envoy\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.116151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.101908    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-token\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.116151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.102082    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-data\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.118462 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.118417    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-data\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.134833 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.134782    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:35.135177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.135151    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-certs\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.135545 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.135525    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-socket\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.135749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.135718    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istiod-ca-cert\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.155936 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.153908    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-certs\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.155936 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.154285    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-socket\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.187328 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.166991    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istiod-ca-cert\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.193681 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.192597    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-envoy\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.193681 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.192891    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-podinfo\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.202117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.195882    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flh6p\" (UniqueName: \"kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-kube-api-access-flh6p\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.203767 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.203709    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-token\") pod \"e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") " pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:35.436742 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.436709    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:35.440753 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:35.440656    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:36.129453 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:10:36.115975    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d9c9bd_494d_42a7_98fc_bb1f11be4762.slice/crio-ba4980ba22bc4ff584858f09442fed8585d56d960c980aeb30a9aaaff63f4359 WatchSource:0}: Error finding container ba4980ba22bc4ff584858f09442fed8585d56d960c980aeb30a9aaaff63f4359: Status 404 returned error can't find the container with id ba4980ba22bc4ff584858f09442fed8585d56d960c980aeb30a9aaaff63f4359
Oct 30 19:10:36.129453 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:36.124410    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"]
Oct 30 19:10:36.137156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:36.137112    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:10:36.137276 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:36.137204    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:10:36.137276 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:36.137252    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:10:36.494121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:36.492746    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" event={"ID":"b2d9c9bd-494d-42a7-98fc-bb1f11be4762","Type":"ContainerStarted","Data":"dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65"}
Oct 30 19:10:36.494121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:36.492816    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" event={"ID":"b2d9c9bd-494d-42a7-98fc-bb1f11be4762","Type":"ContainerStarted","Data":"ba4980ba22bc4ff584858f09442fed8585d56d960c980aeb30a9aaaff63f4359"}
Oct 30 19:10:36.521030 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:36.520422    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" podStartSLOduration=2.5204056120000002 podStartE2EDuration="2.520405612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:36.517551233 +0000 UTC m=+2058.233896310" watchObservedRunningTime="2026-10-30 19:10:36.520405612 +0000 UTC m=+2058.236750690"
Oct 30 19:10:36.673226 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:36.666587    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:37.441707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:37.441668    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:37.448292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:37.448267    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:37.500358 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:37.500329    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:37.501424 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:37.501389    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:10:37.669943 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:37.669911    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:37.851753 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:37.851672    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:38.049125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:38.045986    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:38.540490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:38.540451    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:38.819631 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:38.819060    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:39.282230 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:39.282185    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:39.503882 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:39.503851    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:39.704764 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:39.700199    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:40.491483 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:40.491252    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:40.745178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:40.741063    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:41.653136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:41.651552    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:42.265122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:42.264415    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:43.181232 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:43.181197    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:43.533069 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:10:43.532798    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:10:43.655118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:43.654855    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:43.843694 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:43.843593    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:44.231145 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:44.220925    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:44.412484 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:44.412444    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:45.083405 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:45.083367    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:45.771120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:45.770332    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:46.496163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:46.484824    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:47.131935 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:47.131897    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:47.988153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:47.984238    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:48.596129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:48.593881    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:49.271146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:49.267599    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:49.981124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:49.975706    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:50.457237 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:50.457204    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:50.641766 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:50.641726    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:50.830540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:50.830438    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:51.341887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:51.341853    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:52.264796 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:52.264758    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:52.991017 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:52.990979    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:53.689999 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:53.689960    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:54.383135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:54.383008    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:55.003508 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:55.003428    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:55.817781 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:55.817744    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:56.358425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:56.358008    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:56.538708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:56.538665    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:56.719283 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:56.719251    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:56.966725 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:56.966683    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:57.470607 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:57.470570    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:57.991958 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:57.991922    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:58.760313 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:58.760274    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:59.496121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:59.490663    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:59.674510 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:10:59.674469    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:00.394240 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:00.394205    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:00.968120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:00.967918    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:01.333535 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:01.333439    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:01.575119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:01.558581    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:02.659240 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:02.659210    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:02.849569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:02.849534    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:03.682219 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:03.682166    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:03.927698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:03.911768    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:04.648282 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:04.648244    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:05.148269 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:05.148236    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:05.417161 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:05.417126    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:05.722005 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:05.721909    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:05.911980 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:05.911945    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:06.670806 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:06.670769    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:07.378784 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:07.378748    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:08.120996 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:08.120961    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:08.861678 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:08.861640    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:09.062131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:09.061987    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:10.068706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:10.068673    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:10.988128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:10.985270    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:11.163574 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:11.163534    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:11.749128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:11.749066    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:12.529960 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:12.529927    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:13.166122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:13.165667    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:13.308157 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:11:13.308118    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:11:13.977049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:13.977015    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:14.627220 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:14.627175    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:15.265121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:15.263708    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:15.925131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:15.924653    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:16.646116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:16.645080    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:17.283163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:17.276234    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:17.907692 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:17.907654    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:18.770217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:18.770023    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:19.366820 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:19.364697    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:19.981139 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:19.979903    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:20.687394 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:20.687332    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:21.389916 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:21.389850    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:21.990151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:21.984576    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:22.256755 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:22.256663    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 19:11:22.317206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:22.317183    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 19:11:22.590581 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:22.590497    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:23.258682 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:23.258649    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:23.869025 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:23.868805    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:24.599465 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:24.599433    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:25.277148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:25.277112    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:25.895280 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:25.895241    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:26.581124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:26.579304    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:27.201006 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:27.200167    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:27.888900 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:27.888825    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:28.591176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:28.579992    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:29.324962 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:29.324926    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:30.059685 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:30.059617    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:30.767923 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:30.767890    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:31.393636 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:31.393601    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:32.284120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:32.283544    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:32.953701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:32.951607    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:33.775256 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:33.775219    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:35.374239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:35.374186    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.409229 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:35.409186    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.427350 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:35.427280    4925 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.500120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:35.495229    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.597291 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:35.597246    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.947998 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:35.944305    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:36.847944 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:36.847898    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:37.698406 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:37.698367    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:38.268540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:38.268480    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:38.481984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:38.481939    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:38.711153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:38.711113    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:40.110950 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:40.110893    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:40.944229 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:40.944195    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:41.707427 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:41.707387    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:42.362124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:42.359728    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:43.139152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:43.139059    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:43.306471 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:11:43.306421    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:11:43.776266 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:43.776143    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:44.477238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:44.477206    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:45.183208 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:45.183173    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:45.629404 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:45.629329    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:45.838765 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:45.838731    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:46.860121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:46.855054    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:47.807323 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:47.807283    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:48.550120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:48.546772    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:49.268196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:49.268164    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:50.090361 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:50.090325    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:50.707186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:50.703242    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:51.347855 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:51.347823    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:52.178526 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:52.175143    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:52.791749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:52.789747    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:53.456306 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:53.456087    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:54.144410 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:54.141561    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:54.883664 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:54.883629    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:55.594763 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:55.594666    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:56.203494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:56.203459    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:56.890460 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:56.890250    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:57.716722 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:57.716632    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:58.390120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:58.389892    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:59.084185 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:59.080668    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:59.690158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:11:59.677889    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:00.280737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:00.280700    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:00.902348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:00.902308    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:01.585350 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:01.585293    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:02.303123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:02.297610    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:03.000248 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:03.000163    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:03.615142 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:03.609180    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:04.245407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:04.245360    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:04.450737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:04.450684    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:04.903899 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:04.903754    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:05.101116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:05.101064    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:05.556729 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:05.556646    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:05.796268 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:05.796203    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:05.997366 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:05.997332    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:06.555117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:06.550967    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:07.527117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:07.519873    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:08.671510 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:08.671456    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:09.405523 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:09.405485    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:09.590777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:09.590743    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:10.539122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:10.538777    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:11.149568 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:11.149534    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:11.339714 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:11.339678    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:11.699499 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:11.699464    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:11.994206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:11.984192    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:12.682119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:12.682016    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:13.308972 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:12:13.308933    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:12:13.566959 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:13.566877    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:14.340123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:14.337421    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:15.094280 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:15.094243    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:15.852672 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:15.852115    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:16.483084 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:16.483003    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:17.187608 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:17.187573    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:17.439681 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:17.439571    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:17.648272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:17.648239    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:18.561119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:18.560309    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:18.778122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:18.776488    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:19.474658 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:19.474623    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:20.205144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:20.205090    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:20.879899 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:20.879866    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:21.575812 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:21.575762    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:22.204900 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:22.204796    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:22.942693 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:22.942655    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:23.508698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:23.508664    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:24.285294 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:24.285252    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:24.936568 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:24.936533    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:25.226174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:25.223682    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:26.443742 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:26.443704    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:27.102122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:27.101219    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:28.330817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:28.330782    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:29.080163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:29.080122    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:29.279088 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:29.279048    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:29.479061 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:29.479024    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:29.853631 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:29.853545    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:30.357615 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:30.357576    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:31.086928 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:31.086891    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:31.763773 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:31.763719    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:32.576251 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:32.576216    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:33.303204 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:33.303162    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:33.726370 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:33.726338    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:34.088543 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:34.088457    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:34.833283 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:34.833237    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:35.444432 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:35.444391    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:36.228056 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:36.228015    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:36.530374 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:36.530287    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:36.711559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:36.711523    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:36.974763 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:36.974729    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:37.257798 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:37.257704    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:37.630376 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:37.630284    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:38.290029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:38.289198    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:38.892797 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:38.892763    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:39.082124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:39.078787    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:40.288822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:40.288740    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:40.615452 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:40.615367    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:41.081214 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:41.081166    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:41.769765 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:41.769732    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:42.512854 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:42.512766    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:43.209122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:43.208178    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:43.314834 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:12:43.314798    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:12:43.436775 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:43.436744    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:43.637958 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:43.637853    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:43.936604 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:43.936568    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:44.125353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:44.125302    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:44.338725 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:44.338640    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:44.765292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:44.765058    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:44.955720 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:44.955638    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:45.145655 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:45.145572    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:45.353814 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:45.343856    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:45.640785 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:45.640749    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:45.851825 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:45.851791    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:46.667294 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:46.667260    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:47.399875 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:47.399831    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:47.606130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:47.601715    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:47.790210 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:47.790179    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:47.991089 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:47.991039    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:48.174974 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:48.174941    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:48.368206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:48.368167    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:48.938390 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:48.938354    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:49.592451 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:49.592411    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:50.261045 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:50.251721    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:50.885980 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:50.885944    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:51.683747 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:51.683714    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:51.876150 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:51.876114    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:52.296849 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:52.296814    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:52.587176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:52.581750    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:53.163870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:53.163838    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:53.362356 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:53.362315    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:53.571540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:53.571447    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:54.151452 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:54.151321    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:54.359122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:54.357869    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:54.773384 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:54.773338    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:54.963122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:54.950315    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:55.491467 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:55.491433    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:55.762842 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:55.762754    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:56.488706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:56.488660    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:57.129136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:57.128764    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:57.871846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:57.871810    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:58.498120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:58.493191    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:59.166717 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:59.166679    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:59.862670 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:12:59.862613    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:00.590346 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:00.590302    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:01.448856 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:01.448818    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:01.635543 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:01.635497    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:02.134972 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:02.134935    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:02.337545 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:02.337504    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:03.064125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:03.060996    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:03.793884 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:03.793851    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:04.493773 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:04.493739    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:05.153169 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:05.150052    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:05.774914 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:05.767922    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:06.468123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:06.461727    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:07.169272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:07.163950    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:07.767440 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:07.767404    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:08.439661 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:08.439629    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:08.634082 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:08.634045    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:08.850186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:08.839692    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:09.300143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:09.219172    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:10.193719 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:10.193685    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:10.383159 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:10.383051    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:10.960300 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:10.958912    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:11.212301 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:11.212186    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:11.682559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:11.682521    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:12.540176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:12.540144    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:13.315178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:13.315139    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:13.404671 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:13:13.404613    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:13:13.920086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:13.920048    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:14.701139 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:14.696397    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:15.273948 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:15.273915    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:16.073183 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:16.072788    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:16.801367 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:16.801334    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:17.584558 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:17.584463    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:18.293112 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:18.293060    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:18.921742 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:18.921701    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:19.616610 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:19.616566    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:20.262127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:20.258239    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:20.901066 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:20.901022    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:22.548816 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:22.548780    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:23.071927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:23.071889    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:23.709060 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:23.709023    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:24.489122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:24.484134    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:25.228121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:25.204471    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:25.618314 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:25.618229    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:26.016179 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:26.009407    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:26.684017 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:26.682164    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:27.304122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:27.300381    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:28.041816 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:28.041778    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:28.774463 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:28.774428    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:29.513318 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:29.513170    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:29.592119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:29.585488    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"]
Oct 30 19:13:29.592119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:29.586293    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy" containerID="cri-o://d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92" gracePeriod=30
Oct 30 19:13:30.188280 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:30.188241    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:30.455273 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:30.455197    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"]
Oct 30 19:13:30.455455 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:30.455427    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy" containerID="cri-o://d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1" gracePeriod=30
Oct 30 19:13:30.915177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:30.911324    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:31.099305 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:31.099265    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"]
Oct 30 19:13:31.099528 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:31.099503    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy" containerID="cri-o://32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e" gracePeriod=30
Oct 30 19:13:31.682089 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:31.682047    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:32.481183 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:32.481147    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:33.180127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:33.180007    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:33.890122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:33.884570    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:34.601461 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:34.601332    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:34.936862 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:34.936838    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:13:34.991869 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:34.991831    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"]
Oct 30 19:13:34.993582 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:34.993552    4925 cpu_manager.go:414] "containerMap: removing container" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy"
Oct 30 19:13:34.993750 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:34.993736    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy"
Oct 30 19:13:34.994223 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:34.994202    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="98a19f8f-b107-4093-ac58-115127210713" containerName="istio-proxy"
Oct 30 19:13:35.001251 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.001224    4925 generic.go:356] "Generic (PLEG): container finished" podID="98a19f8f-b107-4093-ac58-115127210713" containerID="d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92" exitCode=0
Oct 30 19:13:35.001568 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.001546    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" event={"ID":"98a19f8f-b107-4093-ac58-115127210713","Type":"ContainerDied","Data":"d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92"}
Oct 30 19:13:35.001714 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.001698    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr" event={"ID":"98a19f8f-b107-4093-ac58-115127210713","Type":"ContainerDied","Data":"f017d4d73923e012a2487b4c02e6cb6f64f8beba27d2312bda26d0f77f2f91e0"}
Oct 30 19:13:35.001832 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.001820    4925 scope.go:122] "RemoveContainer" containerID="d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92"
Oct 30 19:13:35.011472 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.011453    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"
Oct 30 19:13:35.011951 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.011929    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.018171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.017706    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"]
Oct 30 19:13:35.018292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.018226    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-1dd09d-openshift-default-dockercfg-cj522\""
Oct 30 19:13:35.023747 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.023723    4925 scope.go:122] "RemoveContainer" containerID="d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92"
Oct 30 19:13:35.024180 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:13:35.024153    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92\": container with ID starting with d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92 not found: ID does not exist" containerID="d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92"
Oct 30 19:13:35.024348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.024320    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92"} err="failed to get container status \"d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92\": rpc error: code = NotFound desc = could not find container \"d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92\": container with ID starting with d8420e351f5bb8eb3fef640bb70bce5a11a1468bc979aae19e9ffae1eff50d92 not found: ID does not exist"
Oct 30 19:13:35.082859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.082821    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-certs\") pod \"98a19f8f-b107-4093-ac58-115127210713\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") "
Oct 30 19:13:35.084048 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.084020    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-socket\") pod \"98a19f8f-b107-4093-ac58-115127210713\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") "
Oct 30 19:13:35.084353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.084331    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/98a19f8f-b107-4093-ac58-115127210713-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a19f8f-b107-4093-ac58-115127210713-istio-podinfo\") pod \"98a19f8f-b107-4093-ac58-115127210713\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") "
Oct 30 19:13:35.084501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.084484    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98a19f8f-b107-4093-ac58-115127210713-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a19f8f-b107-4093-ac58-115127210713-istiod-ca-cert\") pod \"98a19f8f-b107-4093-ac58-115127210713\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") "
Oct 30 19:13:35.084682 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.084624    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-istio-token\" (UniqueName: \"kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-istio-token\") pod \"98a19f8f-b107-4093-ac58-115127210713\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") "
Oct 30 19:13:35.084842 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.084824    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-kube-api-access-v8d4z\" (UniqueName: \"kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-kube-api-access-v8d4z\") pod \"98a19f8f-b107-4093-ac58-115127210713\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") "
Oct 30 19:13:35.085072 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.085053    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-data\") pod \"98a19f8f-b107-4093-ac58-115127210713\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") "
Oct 30 19:13:35.087079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.086203    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-envoy\") pod \"98a19f8f-b107-4093-ac58-115127210713\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") "
Oct 30 19:13:35.087079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.086302    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-credential-socket\") pod \"98a19f8f-b107-4093-ac58-115127210713\" (UID: \"98a19f8f-b107-4093-ac58-115127210713\") "
Oct 30 19:13:35.087079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.086701    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7hv8\" (UniqueName: \"kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-kube-api-access-t7hv8\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.087079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.086766    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-envoy\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.087079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.086811    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-certs\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.087434 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.087234    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istiod-ca-cert\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.087434 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.087410    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-data\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.087555    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-credential-socket\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.087732    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-podinfo\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.089566    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-socket\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.083928    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-certs" pod "98a19f8f-b107-4093-ac58-115127210713" (UID: "98a19f8f-b107-4093-ac58-115127210713"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.087912    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98a19f8f-b107-4093-ac58-115127210713-istiod-ca-cert" pod "98a19f8f-b107-4093-ac58-115127210713" (UID: "98a19f8f-b107-4093-ac58-115127210713"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.088179    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-socket" pod "98a19f8f-b107-4093-ac58-115127210713" (UID: "98a19f8f-b107-4093-ac58-115127210713"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.088365    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-data" pod "98a19f8f-b107-4093-ac58-115127210713" (UID: "98a19f8f-b107-4093-ac58-115127210713"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.088988    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-istio-token" pod "98a19f8f-b107-4093-ac58-115127210713" (UID: "98a19f8f-b107-4093-ac58-115127210713"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.089387    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-credential-socket" pod "98a19f8f-b107-4093-ac58-115127210713" (UID: "98a19f8f-b107-4093-ac58-115127210713"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.089657    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-token\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.089855    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.089876    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98a19f8f-b107-4093-ac58-115127210713-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.089891    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.089904    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.089917    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:35.089984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.089934    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:35.090973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.090224    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98a19f8f-b107-4093-ac58-115127210713-istio-podinfo" pod "98a19f8f-b107-4093-ac58-115127210713" (UID: "98a19f8f-b107-4093-ac58-115127210713"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:35.091270 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.091244    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-envoy" pod "98a19f8f-b107-4093-ac58-115127210713" (UID: "98a19f8f-b107-4093-ac58-115127210713"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:35.092558 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.092529    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-kube-api-access-v8d4z" pod "98a19f8f-b107-4093-ac58-115127210713" (UID: "98a19f8f-b107-4093-ac58-115127210713"). InnerVolumeSpecName "kube-api-access-v8d4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:35.191270 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.191231    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-credential-socket\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.191498 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.191396    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-podinfo\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.191498 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.191437    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-socket\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.191498 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.191482    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-token\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.191708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.191557    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7hv8\" (UniqueName: \"kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-kube-api-access-t7hv8\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.191708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.191607    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-envoy\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.191708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.191643    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-certs\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.191708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.191696    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-credential-socket\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.191941 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.191805    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-socket\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.191941 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.191843    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istiod-ca-cert\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.192065 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.191986    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-data\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.192065 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.191994    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-certs\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.192258 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.192235    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98a19f8f-b107-4093-ac58-115127210713-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:35.192327 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.192262    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8d4z\" (UniqueName: \"kubernetes.io/projected/98a19f8f-b107-4093-ac58-115127210713-kube-api-access-v8d4z\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:35.192327 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.192281    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98a19f8f-b107-4093-ac58-115127210713-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:35.192327 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.192283    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-data\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.192912 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.192889    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istiod-ca-cert\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.194315 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.194287    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-envoy\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.194696 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.194678    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-podinfo\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.202237 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.202212    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-token\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.202592 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.202572    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7hv8\" (UniqueName: \"kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-kube-api-access-t7hv8\") pod \"e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") " pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.337677 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.336469    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:35.374588 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.368779    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:35.381119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.375820    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"]
Oct 30 19:13:35.402120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.389487    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-9dgtr"]
Oct 30 19:13:35.759283 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.759243    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"]
Oct 30 19:13:35.764250 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:13:35.764207    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf78613e0_ea60_4e84_b6f9_e61564d1cdcc.slice/crio-14b9b73e31483ec6fcd82a1b7e316434594134b0cb0e089d506e4990cffe18ae WatchSource:0}: Error finding container 14b9b73e31483ec6fcd82a1b7e316434594134b0cb0e089d506e4990cffe18ae: Status 404 returned error can't find the container with id 14b9b73e31483ec6fcd82a1b7e316434594134b0cb0e089d506e4990cffe18ae
Oct 30 19:13:35.768953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.768911    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:13:35.769068 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.769005    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:13:35.769068 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.769046    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:13:35.902032 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.901388    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 19:13:35.955122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.943493    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"]
Oct 30 19:13:35.955122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.945118    4925 cpu_manager.go:414] "containerMap: removing container" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy"
Oct 30 19:13:35.955122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.945141    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy"
Oct 30 19:13:35.955122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.945162    4925 cpu_manager.go:414] "containerMap: removing container" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy"
Oct 30 19:13:35.955122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.945170    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy"
Oct 30 19:13:35.955122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.945457    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy"
Oct 30 19:13:35.955122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.945474    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="368eed20-8aa8-4387-a513-6be2459125dc" containerName="istio-proxy"
Oct 30 19:13:35.955122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.954612    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"]
Oct 30 19:13:35.955122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.954722    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:35.977127 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:35.974983    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-4a477d-openshift-default-dockercfg-6bds5\""
Oct 30 19:13:36.024694 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.024603    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-envoy\") pod \"368eed20-8aa8-4387-a513-6be2459125dc\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") "
Oct 30 19:13:36.026369 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.026335    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-istio-token\" (UniqueName: \"kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-istio-token\") pod \"368eed20-8aa8-4387-a513-6be2459125dc\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") "
Oct 30 19:13:36.026504 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.026429    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-data\") pod \"368eed20-8aa8-4387-a513-6be2459125dc\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") "
Oct 30 19:13:36.026591 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.026528    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/368eed20-8aa8-4387-a513-6be2459125dc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368eed20-8aa8-4387-a513-6be2459125dc-istiod-ca-cert\") pod \"368eed20-8aa8-4387-a513-6be2459125dc\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") "
Oct 30 19:13:36.026591 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.026566    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/368eed20-8aa8-4387-a513-6be2459125dc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368eed20-8aa8-4387-a513-6be2459125dc-istio-podinfo\") pod \"368eed20-8aa8-4387-a513-6be2459125dc\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") "
Oct 30 19:13:36.026723 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.026687    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-certs\") pod \"368eed20-8aa8-4387-a513-6be2459125dc\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") "
Oct 30 19:13:36.026788 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.026766    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-credential-socket\") pod \"368eed20-8aa8-4387-a513-6be2459125dc\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") "
Oct 30 19:13:36.027594 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.026910    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-socket\") pod \"368eed20-8aa8-4387-a513-6be2459125dc\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") "
Oct 30 19:13:36.027594 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.026976    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-kube-api-access-c4qt6\" (UniqueName: \"kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-kube-api-access-c4qt6\") pod \"368eed20-8aa8-4387-a513-6be2459125dc\" (UID: \"368eed20-8aa8-4387-a513-6be2459125dc\") "
Oct 30 19:13:36.031409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.027994    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-data" pod "368eed20-8aa8-4387-a513-6be2459125dc" (UID: "368eed20-8aa8-4387-a513-6be2459125dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:36.031409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.028478    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/368eed20-8aa8-4387-a513-6be2459125dc-istiod-ca-cert" pod "368eed20-8aa8-4387-a513-6be2459125dc" (UID: "368eed20-8aa8-4387-a513-6be2459125dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:36.031409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.028500    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-certs" pod "368eed20-8aa8-4387-a513-6be2459125dc" (UID: "368eed20-8aa8-4387-a513-6be2459125dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:36.031409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.028524    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-credential-socket" pod "368eed20-8aa8-4387-a513-6be2459125dc" (UID: "368eed20-8aa8-4387-a513-6be2459125dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:36.031409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.028757    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-socket" pod "368eed20-8aa8-4387-a513-6be2459125dc" (UID: "368eed20-8aa8-4387-a513-6be2459125dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:36.031409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.030247    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-credential-socket\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.031409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.030488    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-podinfo\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.031409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.030841    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-token\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.031409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.030877    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-socket\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.031409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.030923    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-envoy\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.031409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.031278    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istiod-ca-cert\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.031409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.031322    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnljv\" (UniqueName: \"kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-kube-api-access-vnljv\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.032212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.031730    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-data\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.032212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.031800    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-certs\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.032212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.031929    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:36.032212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.031947    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368eed20-8aa8-4387-a513-6be2459125dc-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:36.032212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.031963    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:36.032212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.031979    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:36.032212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.031992    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:36.076122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.049323    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/368eed20-8aa8-4387-a513-6be2459125dc-istio-podinfo" pod "368eed20-8aa8-4387-a513-6be2459125dc" (UID: "368eed20-8aa8-4387-a513-6be2459125dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:36.076122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.049455    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-istio-token" pod "368eed20-8aa8-4387-a513-6be2459125dc" (UID: "368eed20-8aa8-4387-a513-6be2459125dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:36.076122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.049666    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-kube-api-access-c4qt6" pod "368eed20-8aa8-4387-a513-6be2459125dc" (UID: "368eed20-8aa8-4387-a513-6be2459125dc"). InnerVolumeSpecName "kube-api-access-c4qt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:36.113119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.088304    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-envoy" pod "368eed20-8aa8-4387-a513-6be2459125dc" (UID: "368eed20-8aa8-4387-a513-6be2459125dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.187242    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnljv\" (UniqueName: \"kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-kube-api-access-vnljv\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.187409    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-data\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.187452    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-certs\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.187502    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-credential-socket\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.187625    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-podinfo\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.187894    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-data\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.187931    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-token\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.187972    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-socket\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.188019    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-envoy\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.188176    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istiod-ca-cert\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.188305    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368eed20-8aa8-4387-a513-6be2459125dc-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.188324    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4qt6\" (UniqueName: \"kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-kube-api-access-c4qt6\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.188340    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368eed20-8aa8-4387-a513-6be2459125dc-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.188356    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368eed20-8aa8-4387-a513-6be2459125dc-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.189124    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istiod-ca-cert\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.189566    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.189684    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-socket\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.191552    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-podinfo\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.192474    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-credential-socket\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.193438    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" event={"ID":"f78613e0-ea60-4e84-b6f9-e61564d1cdcc","Type":"ContainerStarted","Data":"7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5"}
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.193498    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" event={"ID":"f78613e0-ea60-4e84-b6f9-e61564d1cdcc","Type":"ContainerStarted","Data":"14b9b73e31483ec6fcd82a1b7e316434594134b0cb0e089d506e4990cffe18ae"}
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.196088    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-certs\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.198685    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnljv\" (UniqueName: \"kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-kube-api-access-vnljv\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.200040    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-token\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.205597    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-envoy\") pod \"e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") " pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.206404    4925 generic.go:356] "Generic (PLEG): container finished" podID="368eed20-8aa8-4387-a513-6be2459125dc" containerID="d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1" exitCode=0
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.206454    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.206517    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" event={"ID":"368eed20-8aa8-4387-a513-6be2459125dc","Type":"ContainerDied","Data":"d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1"}
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.206558    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d" event={"ID":"368eed20-8aa8-4387-a513-6be2459125dc","Type":"ContainerDied","Data":"0835f057a6b8d13ab3067f0625bbd300f4482a6c1155a1566fd3f01bff70ac15"}
Oct 30 19:13:36.219038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.206587    4925 scope.go:122] "RemoveContainer" containerID="d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1"
Oct 30 19:13:36.237693 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.237003    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" podStartSLOduration=107.236983598 podStartE2EDuration="1m47.236983598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:36.231477953 +0000 UTC m=+2237.947823055" watchObservedRunningTime="2026-10-30 19:13:36.236983598 +0000 UTC m=+2237.953328717"
Oct 30 19:13:36.255786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.254555    4925 scope.go:122] "RemoveContainer" containerID="37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714"
Oct 30 19:13:36.287659 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.287507    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"]
Oct 30 19:13:36.298055 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.297666    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:36.300776 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.300665    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-5hp6d"]
Oct 30 19:13:36.339530 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.337009    4925 scope.go:122] "RemoveContainer" containerID="d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1"
Oct 30 19:13:36.339530 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.337449    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:36.339530 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:13:36.337582    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1\": container with ID starting with d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1 not found: ID does not exist" containerID="d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1"
Oct 30 19:13:36.339530 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.337614    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1"} err="failed to get container status \"d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1\": rpc error: code = NotFound desc = could not find container \"d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1\": container with ID starting with d6056f3c24e723b6b5464a815aee34d867e24c10c7b0c470966e067420f17ee1 not found: ID does not exist"
Oct 30 19:13:36.339530 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.337638    4925 scope.go:122] "RemoveContainer" containerID="37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714"
Oct 30 19:13:36.340507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.340033    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused" start-of-body=
Oct 30 19:13:36.340507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.340083    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused"
Oct 30 19:13:36.340507 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:13:36.340429    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714\": container with ID starting with 37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714 not found: ID does not exist" containerID="37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714"
Oct 30 19:13:36.340507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.340468    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714"} err="failed to get container status \"37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714\": rpc error: code = NotFound desc = could not find container \"37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714\": container with ID starting with 37e525976886eb8c5707f140c8e0ec848101494e8e73dee3609c9796ac992714 not found: ID does not exist"
Oct 30 19:13:36.372502 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.372394    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:36.448414 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.446661    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="368eed20-8aa8-4387-a513-6be2459125dc" path="/var/lib/kubelet/pods/368eed20-8aa8-4387-a513-6be2459125dc/volumes"
Oct 30 19:13:36.448414 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.447643    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98a19f8f-b107-4093-ac58-115127210713" path="/var/lib/kubelet/pods/98a19f8f-b107-4093-ac58-115127210713/volumes"
Oct 30 19:13:36.983917 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:36.712911    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:37.007461 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.007236    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:37.072921 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.072527    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:13:37.216707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.216288    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-certs\") pod \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") "
Oct 30 19:13:37.216707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.216410    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-envoy\") pod \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") "
Oct 30 19:13:37.216707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.216480    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-credential-socket\") pod \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") "
Oct 30 19:13:37.216707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.216651    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istiod-ca-cert\") pod \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") "
Oct 30 19:13:37.217110 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.216762    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-podinfo\") pod \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") "
Oct 30 19:13:37.217110 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.216798    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-token\" (UniqueName: \"kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-token\") pod \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") "
Oct 30 19:13:37.217110 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.216887    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-kube-api-access-9dq5v\" (UniqueName: \"kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-kube-api-access-9dq5v\") pod \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") "
Oct 30 19:13:37.217110 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.217013    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-socket\") pod \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") "
Oct 30 19:13:37.217398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.217156    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-data\") pod \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\" (UID: \"33bb4a2a-8455-46ae-b048-29f7ba08eef7\") "
Oct 30 19:13:37.218051 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.218013    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-certs" pod "33bb4a2a-8455-46ae-b048-29f7ba08eef7" (UID: "33bb4a2a-8455-46ae-b048-29f7ba08eef7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:37.225555 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.225518    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-data" pod "33bb4a2a-8455-46ae-b048-29f7ba08eef7" (UID: "33bb4a2a-8455-46ae-b048-29f7ba08eef7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:37.226528 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.225756    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-socket" pod "33bb4a2a-8455-46ae-b048-29f7ba08eef7" (UID: "33bb4a2a-8455-46ae-b048-29f7ba08eef7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:37.232570 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.232542    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:37.232689 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.232652    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:37.232689 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.232675    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:37.235734 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.235602    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istiod-ca-cert" pod "33bb4a2a-8455-46ae-b048-29f7ba08eef7" (UID: "33bb4a2a-8455-46ae-b048-29f7ba08eef7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:37.237223 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.236512    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-credential-socket" pod "33bb4a2a-8455-46ae-b048-29f7ba08eef7" (UID: "33bb4a2a-8455-46ae-b048-29f7ba08eef7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:37.244585 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.244548    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-envoy" pod "33bb4a2a-8455-46ae-b048-29f7ba08eef7" (UID: "33bb4a2a-8455-46ae-b048-29f7ba08eef7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:37.246376 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.246242    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-kube-api-access-9dq5v" pod "33bb4a2a-8455-46ae-b048-29f7ba08eef7" (UID: "33bb4a2a-8455-46ae-b048-29f7ba08eef7"). InnerVolumeSpecName "kube-api-access-9dq5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:37.246774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.246749    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-token" pod "33bb4a2a-8455-46ae-b048-29f7ba08eef7" (UID: "33bb4a2a-8455-46ae-b048-29f7ba08eef7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:37.250860 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.250207    4925 generic.go:356] "Generic (PLEG): container finished" podID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerID="32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e" exitCode=0
Oct 30 19:13:37.250860 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.250434    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" event={"ID":"33bb4a2a-8455-46ae-b048-29f7ba08eef7","Type":"ContainerDied","Data":"32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e"}
Oct 30 19:13:37.250860 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.250528    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4" event={"ID":"33bb4a2a-8455-46ae-b048-29f7ba08eef7","Type":"ContainerDied","Data":"a751bb2c33a6330b7a4d015d60cea96374cfadf78604561d37a2e284d4235b0a"}
Oct 30 19:13:37.250860 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.250566    4925 scope.go:122] "RemoveContainer" containerID="32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e"
Oct 30 19:13:37.250860 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.250763    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"
Oct 30 19:13:37.255039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.254877    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-podinfo" pod "33bb4a2a-8455-46ae-b048-29f7ba08eef7" (UID: "33bb4a2a-8455-46ae-b048-29f7ba08eef7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:13:37.294759 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.292273    4925 scope.go:122] "RemoveContainer" containerID="32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e"
Oct 30 19:13:37.294759 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:13:37.292667    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e\": container with ID starting with 32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e not found: ID does not exist" containerID="32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e"
Oct 30 19:13:37.294759 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.292707    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e"} err="failed to get container status \"32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e\": rpc error: code = NotFound desc = could not find container \"32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e\": container with ID starting with 32ddae121ae2b25f7f326a7dbdd4aeaca1627ddf1b19e36c7a52a3015c3b4b8e not found: ID does not exist"
Oct 30 19:13:37.301132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.299058    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:13:37.301132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.299160    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:13:37.301132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.299208    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:13:37.302216 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.302196    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"]
Oct 30 19:13:37.334407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.334368    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dq5v\" (UniqueName: \"kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-kube-api-access-9dq5v\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:37.334407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.334412    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:37.334767 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.334428    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33bb4a2a-8455-46ae-b048-29f7ba08eef7-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:37.334767 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.334443    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:37.334767 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.334458    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:37.334767 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.334473    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33bb4a2a-8455-46ae-b048-29f7ba08eef7-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:37.339191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.339158    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused" start-of-body=
Oct 30 19:13:37.339330 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.339218    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused"
Oct 30 19:13:37.608242 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.608195    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"]
Oct 30 19:13:37.608545 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.608517    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-857g4"]
Oct 30 19:13:37.733064 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:37.731183    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:38.295913 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:38.295868    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" event={"ID":"532cf3a3-18af-48ac-8df2-ce52f16f98ea","Type":"ContainerStarted","Data":"cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4"}
Oct 30 19:13:38.296515 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:38.295954    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" event={"ID":"532cf3a3-18af-48ac-8df2-ce52f16f98ea","Type":"ContainerStarted","Data":"9a742c0ac2606f538214325454045185c18ac6bd1aeed3dc220dff706b58a81a"}
Oct 30 19:13:38.298902 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:38.298880    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:38.301349 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:38.301316    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused" start-of-body=
Oct 30 19:13:38.301516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:38.301372    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" podUID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused"
Oct 30 19:13:38.337193 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:38.337160    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused" start-of-body=
Oct 30 19:13:38.337314 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:38.337224    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused"
Oct 30 19:13:38.341926 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:38.341870    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" podStartSLOduration=101.341854707 podStartE2EDuration="1m41.341854707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:38.340657853 +0000 UTC m=+2240.057002931" watchObservedRunningTime="2026-10-30 19:13:38.341854707 +0000 UTC m=+2240.058199786"
Oct 30 19:13:38.428558 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:38.428525    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" path="/var/lib/kubelet/pods/33bb4a2a-8455-46ae-b048-29f7ba08eef7/volumes"
Oct 30 19:13:38.900006 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:38.899969    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:39.312125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:39.306529    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused" start-of-body=
Oct 30 19:13:39.312125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:39.306603    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" podUID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused"
Oct 30 19:13:39.337085 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:39.337057    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused" start-of-body=
Oct 30 19:13:39.337344 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:39.337321    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused"
Oct 30 19:13:39.762853 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:39.762751    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:40.335706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:40.335644    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused" start-of-body=
Oct 30 19:13:40.336323 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:40.335703    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" podUID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused"
Oct 30 19:13:40.337296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:40.337218    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused" start-of-body=
Oct 30 19:13:40.337296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:40.337259    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused"
Oct 30 19:13:40.796167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:40.787958    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:41.313122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:41.312485    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused" start-of-body=
Oct 30 19:13:41.313122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:41.312552    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" podUID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused"
Oct 30 19:13:41.337731 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:41.337562    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused" start-of-body=
Oct 30 19:13:41.337731 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:41.337613    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused"
Oct 30 19:13:41.707206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:41.706835    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:42.306120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:42.305480    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused" start-of-body=
Oct 30 19:13:42.306120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:42.305547    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" podUID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused"
Oct 30 19:13:42.362121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:42.360331    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused" start-of-body=
Oct 30 19:13:42.362121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:42.360399    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused"
Oct 30 19:13:42.886128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:42.885789    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:43.298860 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:43.298816    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused" start-of-body=
Oct 30 19:13:43.299054 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:43.298897    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" podUID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused"
Oct 30 19:13:43.319768 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:13:43.319725    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:13:43.337936 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:43.337910    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused" start-of-body=
Oct 30 19:13:43.338136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:43.337959    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.234:15021/healthz/ready\": dial tcp 10.132.0.234:15021: connect: connection refused"
Oct 30 19:13:43.827133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:43.813203    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:44.151240 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:44.141457    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:44.298456 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:44.298422    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused" start-of-body=
Oct 30 19:13:44.298660 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:44.298481    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" podUID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.235:15021/healthz/ready\": dial tcp 10.132.0.235:15021: connect: connection refused"
Oct 30 19:13:44.329670 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:44.329636    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:44.342833 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:44.342807    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:44.343296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:44.343272    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:44.343998 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:44.343977    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:13:44.979915 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:44.978621    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:45.304528 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:45.304448    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:45.304999 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:45.304973    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:45.305807 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:45.305785    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:13:45.720818 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:45.720784    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:46.476716 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:46.476677    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:47.167067 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:47.167029    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:47.889839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:47.889797    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:48.175608 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:48.175566    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"]
Oct 30 19:13:48.190136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:48.190106    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"]
Oct 30 19:13:48.330977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:48.330943    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:48.524199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:48.516960    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:48.772881 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:48.772840    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:49.477824 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:49.477786    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:50.571444 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:50.571413    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:50.809720 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:50.809684    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:51.002341 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:51.002311    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:51.252887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:51.252805    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:51.450593 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:51.450560    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:51.676756 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:51.676723    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:51.902502 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:51.902465    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:52.322230 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:52.322198    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:52.531378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:52.531343    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:53.127171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.127142    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:53.140140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.140071    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"]
Oct 30 19:13:53.141396 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.141372    4925 cpu_manager.go:414] "containerMap: removing container" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy"
Oct 30 19:13:53.141396 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.141393    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy"
Oct 30 19:13:53.141614 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.141602    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="33bb4a2a-8455-46ae-b048-29f7ba08eef7" containerName="istio-proxy"
Oct 30 19:13:53.151807 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.151780    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.181982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.174929    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-ca2cd5\"/\"agc-cred-abc7c59e7c75\""
Oct 30 19:13:53.181982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.175066    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-ca2cd5\"/\"payload-processing-external-model-e2e--3c4018c9-dockercfg-fgjm9\""
Oct 30 19:13:53.181982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.176021    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-ca2cd5\"/\"payload-processing-external-model-plugins-e2e-extmod-a-ca2cd5\""
Oct 30 19:13:53.181982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.176120    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-ca2cd5\"/\"kube-root-ca.crt\""
Oct 30 19:13:53.181982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.176171    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-ca2cd5\"/\"openshift-service-ca.crt\""
Oct 30 19:13:53.181982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.176206    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f" podUID="be5cb88e-59da-4585-9114-484c0773dc19" containerName="payload-processing" containerID="cri-o://cdc8d730ff1a66d2db09bd79f45897d1f4d57f789af184aa7ac45bf880c94abe" gracePeriod=25
Oct 30 19:13:53.181982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.176259    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-ca2cd5\"/\"routing-overlay\""
Oct 30 19:13:53.210841 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.190413    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7" podUID="fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae" containerName="payload-pre-processing" containerID="cri-o://10cca33001b0028e19009cd7cea77fceb3792b6db6d6042004f14a6bc2892721" gracePeriod=25
Oct 30 19:13:53.210841 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.200238    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"]
Oct 30 19:13:53.454397 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.389562    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") " pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.454911 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.421746    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:53.454911 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.454805    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") " pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.454911 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.454854    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v75z\" (UniqueName: \"kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-kube-api-access-2v75z\") pod \"payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") " pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.455244 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.454930    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") " pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.589169 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.557129    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") " pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.589169 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.557219    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") " pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.589169 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.557248    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v75z\" (UniqueName: \"kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-kube-api-access-2v75z\") pod \"payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") " pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.589169 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.557307    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") " pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.590088 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.590055    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") " pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.628140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.592728    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") " pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.704124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.702400    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v75z\" (UniqueName: \"kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-kube-api-access-2v75z\") pod \"payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") " pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.853691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.853617    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") " pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:53.903822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.903454    4925 generic.go:356] "Generic (PLEG): container finished" podID="be5cb88e-59da-4585-9114-484c0773dc19" containerID="cdc8d730ff1a66d2db09bd79f45897d1f4d57f789af184aa7ac45bf880c94abe" exitCode=0
Oct 30 19:13:53.903822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.903661    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f" event={"ID":"be5cb88e-59da-4585-9114-484c0773dc19","Type":"ContainerDied","Data":"cdc8d730ff1a66d2db09bd79f45897d1f4d57f789af184aa7ac45bf880c94abe"}
Oct 30 19:13:53.906600 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.906360    4925 generic.go:356] "Generic (PLEG): container finished" podID="fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae" containerID="10cca33001b0028e19009cd7cea77fceb3792b6db6d6042004f14a6bc2892721" exitCode=0
Oct 30 19:13:53.906600 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:53.906538    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7" event={"ID":"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae","Type":"ContainerDied","Data":"10cca33001b0028e19009cd7cea77fceb3792b6db6d6042004f14a6bc2892721"}
Oct 30 19:13:54.079121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.076356    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:54.120393 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.108441    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"
Oct 30 19:13:54.247001 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.246964    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be5cb88e-59da-4585-9114-484c0773dc19-kube-api-access-zhqmh\" (UniqueName: \"kubernetes.io/projected/be5cb88e-59da-4585-9114-484c0773dc19-kube-api-access-zhqmh\") pod \"be5cb88e-59da-4585-9114-484c0773dc19\" (UID: \"be5cb88e-59da-4585-9114-484c0773dc19\") "
Oct 30 19:13:54.247534 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.247289    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be5cb88e-59da-4585-9114-484c0773dc19-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be5cb88e-59da-4585-9114-484c0773dc19-plugins-config-volume\") pod \"be5cb88e-59da-4585-9114-484c0773dc19\" (UID: \"be5cb88e-59da-4585-9114-484c0773dc19\") "
Oct 30 19:13:54.247790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.247732    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be5cb88e-59da-4585-9114-484c0773dc19-plugins-config-volume" pod "be5cb88e-59da-4585-9114-484c0773dc19" (UID: "be5cb88e-59da-4585-9114-484c0773dc19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:54.248158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.247986    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be5cb88e-59da-4585-9114-484c0773dc19-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:54.249498 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.249472    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be5cb88e-59da-4585-9114-484c0773dc19-kube-api-access-zhqmh" pod "be5cb88e-59da-4585-9114-484c0773dc19" (UID: "be5cb88e-59da-4585-9114-484c0773dc19"). InnerVolumeSpecName "kube-api-access-zhqmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:54.311306 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.310827    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"
Oct 30 19:13:54.349864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.349827    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhqmh\" (UniqueName: \"kubernetes.io/projected/be5cb88e-59da-4585-9114-484c0773dc19-kube-api-access-zhqmh\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:54.450922 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.450893    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-kube-api-access-qgpxl\" (UniqueName: \"kubernetes.io/projected/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-kube-api-access-qgpxl\") pod \"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae\" (UID: \"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae\") "
Oct 30 19:13:54.451083 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.451065    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-plugins-config-volume\") pod \"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae\" (UID: \"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae\") "
Oct 30 19:13:54.451391 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.451370    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-plugins-config-volume" pod "fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae" (UID: "fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:54.451601 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.451586    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:54.453304 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.453281    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-kube-api-access-qgpxl" pod "fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae" (UID: "fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae"). InnerVolumeSpecName "kube-api-access-qgpxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:54.537273 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.532656    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:54.576132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.575288    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgpxl\" (UniqueName: \"kubernetes.io/projected/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae-kube-api-access-qgpxl\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:54.630154 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:13:54.624331    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c8263f_5c5f_4cce_adcb_8e15e22a8158.slice/crio-78a0fdb023ce5f39d582701d4e6a1e9e616c53fd2f16b1ded73ead96aef86cf4 WatchSource:0}: Error finding container 78a0fdb023ce5f39d582701d4e6a1e9e616c53fd2f16b1ded73ead96aef86cf4: Status 404 returned error can't find the container with id 78a0fdb023ce5f39d582701d4e6a1e9e616c53fd2f16b1ded73ead96aef86cf4
Oct 30 19:13:54.646216 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.643221    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"]
Oct 30 19:13:54.923189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.923155    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"
Oct 30 19:13:54.923403 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.923155    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7" event={"ID":"fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae","Type":"ContainerDied","Data":"0fc23c27b0c93effe72b3d9701d55de07a2afe43f1b0ac9262f51e85ce337f88"}
Oct 30 19:13:54.923403 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.923292    4925 scope.go:122] "RemoveContainer" containerID="10cca33001b0028e19009cd7cea77fceb3792b6db6d6042004f14a6bc2892721"
Oct 30 19:13:54.927023 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.926990    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l" event={"ID":"82c8263f-5c5f-4cce-adcb-8e15e22a8158","Type":"ContainerStarted","Data":"18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e"}
Oct 30 19:13:54.927183 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.927049    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:13:54.927183 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.927065    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l" event={"ID":"82c8263f-5c5f-4cce-adcb-8e15e22a8158","Type":"ContainerStarted","Data":"78a0fdb023ce5f39d582701d4e6a1e9e616c53fd2f16b1ded73ead96aef86cf4"}
Oct 30 19:13:54.931287 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.931259    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f" event={"ID":"be5cb88e-59da-4585-9114-484c0773dc19","Type":"ContainerDied","Data":"48705cbdcd185dbc28021f18a27dab40f1ff328f46ec7ea01a334ab6ca957274"}
Oct 30 19:13:54.931396 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.931349    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"
Oct 30 19:13:54.941238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.941185    4925 scope.go:122] "RemoveContainer" containerID="cdc8d730ff1a66d2db09bd79f45897d1f4d57f789af184aa7ac45bf880c94abe"
Oct 30 19:13:55.007401 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:54.999790    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l" podStartSLOduration=1.999770112 podStartE2EDuration="1.999770112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:13:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:54.950846772 +0000 UTC m=+2256.667191849" watchObservedRunningTime="2026-10-30 19:13:54.999770112 +0000 UTC m=+2256.716115189"
Oct 30 19:13:55.007401 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:55.001764    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"]
Oct 30 19:13:55.010568 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:55.010540    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-37f2b8-d57c6cb9f-wd9s7"]
Oct 30 19:13:55.015364 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:55.015335    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"]
Oct 30 19:13:55.019276 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:55.019253    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-37f2b8-7797c9b5d8-qln5f"]
Oct 30 19:13:55.644119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:55.643264    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:56.493142 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:56.493056    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:56.502739 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:56.502705    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be5cb88e-59da-4585-9114-484c0773dc19" path="/var/lib/kubelet/pods/be5cb88e-59da-4585-9114-484c0773dc19/volumes"
Oct 30 19:13:56.503324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:56.503295    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae" path="/var/lib/kubelet/pods/fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae/volumes"
Oct 30 19:13:57.202042 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:57.201993    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:57.906888 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:57.906851    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:58.664842 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:58.664807    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:59.501151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:13:59.498342    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:00.161545 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:00.159254    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:00.784908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:00.784870    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:01.465118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.464869    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:01.554137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.554089    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"]
Oct 30 19:14:01.555041 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.555025    4925 cpu_manager.go:414] "containerMap: removing container" podUID="fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae" containerName="payload-pre-processing"
Oct 30 19:14:01.555119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.555043    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae" containerName="payload-pre-processing"
Oct 30 19:14:01.555119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.555083    4925 cpu_manager.go:414] "containerMap: removing container" podUID="be5cb88e-59da-4585-9114-484c0773dc19" containerName="payload-processing"
Oct 30 19:14:01.555119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.555088    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be5cb88e-59da-4585-9114-484c0773dc19" containerName="payload-processing"
Oct 30 19:14:01.555270 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.555259    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="fcf76ae7-b79b-4c9c-ab65-c88bb2f745ae" containerName="payload-pre-processing"
Oct 30 19:14:01.555309 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.555278    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="be5cb88e-59da-4585-9114-484c0773dc19" containerName="payload-processing"
Oct 30 19:14:01.559411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.559389    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.568437 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.568415    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-d6f956-openshift-default-dockercfg-jmqmf\""
Oct 30 19:14:01.573944 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.573919    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"]
Oct 30 19:14:01.713440 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.713401    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-socket\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.713622 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.713466    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-credential-socket\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.713622 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.713593    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-token\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.713700 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.713627    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-envoy\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.713768 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.713746    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-certs\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.713884 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.713872    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-data\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.713922 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.713901    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e69194e-043b-48dc-aa47-4f6d378a09a4-istiod-ca-cert\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.713958 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.713947    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9v54\" (UniqueName: \"kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-kube-api-access-t9v54\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.714191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.714170    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-podinfo\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.816310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816222    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-podinfo\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.816310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816269    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-socket\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.816310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816311    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-credential-socket\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.816610 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816368    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-token\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.816610 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816389    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-envoy\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.816610 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816476    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-certs\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.816801 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816610    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-data\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.816801 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816654    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e69194e-043b-48dc-aa47-4f6d378a09a4-istiod-ca-cert\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.816801 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816726    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9v54\" (UniqueName: \"kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-kube-api-access-t9v54\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.816801 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816739    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-socket\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.817029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816922    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-certs\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.817029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816969    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-credential-socket\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.817029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.816978    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-data\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.817523 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.817500    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e69194e-043b-48dc-aa47-4f6d378a09a4-istiod-ca-cert\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.819122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.819078    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-envoy\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.819237 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.819125    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-podinfo\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.827683 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.827657    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-token\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.827796 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.827745    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9v54\" (UniqueName: \"kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-kube-api-access-t9v54\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:01.883407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:01.883366    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:02.006845 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:02.006804    4925 generic.go:356] "Generic (PLEG): container finished" podID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerID="dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65" exitCode=0
Oct 30 19:14:02.007020 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:02.006910    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" event={"ID":"b2d9c9bd-494d-42a7-98fc-bb1f11be4762","Type":"ContainerDied","Data":"dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65"}
Oct 30 19:14:02.007574 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:02.007549    4925 scope.go:122] "RemoveContainer" containerID="dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65"
Oct 30 19:14:02.010990 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:02.010944    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:14:02.011158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:02.011048    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:14:02.011158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:02.011145    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:14:02.190117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:02.187951    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:02.287444 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:02.287387    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:14:02.287710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:02.287674    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:14:02.287873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:02.287854    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:14:02.293377 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:02.293354    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"]
Oct 30 19:14:02.991168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:02.987697    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:03.048123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:03.042910    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" event={"ID":"b2d9c9bd-494d-42a7-98fc-bb1f11be4762","Type":"ContainerStarted","Data":"ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0"}
Oct 30 19:14:03.051135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:03.050272    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" event={"ID":"5e69194e-043b-48dc-aa47-4f6d378a09a4","Type":"ContainerStarted","Data":"f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf"}
Oct 30 19:14:03.051135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:03.050336    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" event={"ID":"5e69194e-043b-48dc-aa47-4f6d378a09a4","Type":"ContainerStarted","Data":"1808e0165d2151be08be2da0c0cef823d1f572e0de0496803949539881d23741"}
Oct 30 19:14:03.123223 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:03.123170    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" podStartSLOduration=2.123152308 podStartE2EDuration="2.123152308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:03.119528583 +0000 UTC m=+2264.835873670" watchObservedRunningTime="2026-10-30 19:14:03.123152308 +0000 UTC m=+2264.839497449"
Oct 30 19:14:03.441923 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:03.441815    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:14:03.442745 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:03.442719    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused" start-of-body=
Oct 30 19:14:03.442873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:03.442776    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused"
Oct 30 19:14:03.865172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:03.861982    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:03.897120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:03.892114    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:03.902974 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:03.901226    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused" start-of-body=
Oct 30 19:14:03.902974 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:03.901287    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused"
Oct 30 19:14:04.442144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:04.442111    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused" start-of-body=
Oct 30 19:14:04.442575 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:04.442159    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused"
Oct 30 19:14:04.782749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:04.782660    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:04.884611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:04.884568    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused" start-of-body=
Oct 30 19:14:04.884910 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:04.884879    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused"
Oct 30 19:14:05.441500 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:05.441462    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused" start-of-body=
Oct 30 19:14:05.441682 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:05.441512    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused"
Oct 30 19:14:05.611368 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:05.611336    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:05.893176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:05.885460    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused" start-of-body=
Oct 30 19:14:05.893176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:05.885524    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused"
Oct 30 19:14:05.945980 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:05.945951    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:14:06.335035 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:06.334556    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:06.465612 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:06.465572    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused" start-of-body=
Oct 30 19:14:06.465910 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:06.465877    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused"
Oct 30 19:14:06.918067 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:06.865539    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:06.921130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:06.918831    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused" start-of-body=
Oct 30 19:14:06.921130 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:06.918886    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused"
Oct 30 19:14:07.082154 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:07.081722    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:07.302076 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:07.299646    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:07.441802 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:07.441765    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused" start-of-body=
Oct 30 19:14:07.442016 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:07.441819    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused"
Oct 30 19:14:07.503233 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:07.503197    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:14:07.527857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:07.527760    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:07.899121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:07.887984    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused" start-of-body=
Oct 30 19:14:07.899121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:07.888051    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused"
Oct 30 19:14:08.450534 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:08.450494    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused" start-of-body=
Oct 30 19:14:08.451180 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:08.451146    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused"
Oct 30 19:14:08.884579 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:08.884484    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused" start-of-body=
Oct 30 19:14:08.884866 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:08.884833    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused"
Oct 30 19:14:09.018121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:09.012086    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:09.448132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:09.443196    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:14:09.477121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:09.450999    4925 patch_prober.go:29] interesting pod/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused" start-of-body=
Oct 30 19:14:09.477121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:09.451058    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.233:15021/healthz/ready\": dial tcp 10.132.0.233:15021: connect: connection refused"
Oct 30 19:14:09.884361 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:09.884284    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused" start-of-body=
Oct 30 19:14:09.884511 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:09.884355    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.237:15021/healthz/ready\": dial tcp 10.132.0.237:15021: connect: connection refused"
Oct 30 19:14:10.192825 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:10.125218    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:10.447882 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:10.447788    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:14:10.449363 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:10.449318    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:14:10.889525 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:10.889444    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:10.890012 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:10.889962    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:10.890982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:10.890962    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:14:10.969217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:10.969007    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:11.296988 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:11.296952    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:11.673661 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:11.673629    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:12.382199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:12.382164    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:13.253507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:13.253467    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:13.312023 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:14:13.311986    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:14:13.894667 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:13.894630    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:14.095843 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:14.095811    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:14.326514 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:14.313362    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:15.114608 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:15.114570    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:15.882614 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:15.882572    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:16.522122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:16.521910    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:17.235123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:17.228594    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:17.888850 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:17.888807    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:18.608180 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:18.607112    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:19.213247 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:19.212954    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:19.880994 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:19.880954    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:20.549531 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:20.549496    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:21.170382 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:21.170339    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:21.968573 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:21.968518    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:22.420985 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:22.420949    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:22.656503 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:22.656470    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:22.989365 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:22.972278    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:23.264521 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:23.264474    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:23.458973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:23.458924    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:23.652171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:23.650327    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:24.412650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:24.412610    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:24.588162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:24.588088    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:24.815918 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:24.814725    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:25.126811 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:25.126731    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:25.331869 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:25.331837    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:26.561288 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:26.561256    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:26.745030 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:26.744998    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:27.436711 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:27.436676    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:28.075753 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:28.075714    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:28.793612 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:28.793519    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:29.403119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:29.399270    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:30.096661 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:30.096622    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:30.761708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:30.761662    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:31.583422 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:31.583382    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:31.770665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:31.770629    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:32.970898 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:32.970865    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:33.168382 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:33.168350    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:33.251805 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:33.251723    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"]
Oct 30 19:14:33.265472 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:33.265436    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"]
Oct 30 19:14:33.419238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:33.419204    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:33.621084 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:33.621002    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:33.921222 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:33.921185    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:34.153122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:34.149474    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:34.395684 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:34.395595    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:34.743340 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:34.743306    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:34.979341 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:34.979289    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:35.583617 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:35.583574    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:36.315454 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:36.314705    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:36.967708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:36.967670    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:37.517329 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:37.517296    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"]
Oct 30 19:14:37.517590 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:37.517549    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg" podUID="50dfb53c-b7bf-43eb-8956-10836baafa25" containerName="maas-api" containerID="cri-o://e24dad0f0607e1ba5266f197af29c602995e7368dff97db72fbf0693138cee3d" gracePeriod=30
Oct 30 19:14:37.651376 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:37.651338    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:37.851053 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:37.851015    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:37.960298 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:37.960257    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg" podUID="50dfb53c-b7bf-43eb-8956-10836baafa25" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.208:8443/health\": dial tcp 10.132.0.208:8443: connect: connection refused"
Oct 30 19:14:38.169176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.103605    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:38.193317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.175876    4925 generic.go:356] "Generic (PLEG): container finished" podID="50dfb53c-b7bf-43eb-8956-10836baafa25" containerID="e24dad0f0607e1ba5266f197af29c602995e7368dff97db72fbf0693138cee3d" exitCode=0
Oct 30 19:14:38.195573 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.175905    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg" event={"ID":"50dfb53c-b7bf-43eb-8956-10836baafa25","Type":"ContainerDied","Data":"e24dad0f0607e1ba5266f197af29c602995e7368dff97db72fbf0693138cee3d"}
Oct 30 19:14:38.254234 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.254187    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m" podUID="13c22a74-d32d-4400-bde2-eb77c344c8b9" containerName="payload-processing" containerID="cri-o://7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2" gracePeriod=25
Oct 30 19:14:38.265968 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.265913    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb" podUID="cfe122bd-8ed6-4a29-8931-8b93ef12f26b" containerName="payload-pre-processing" containerID="cri-o://1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596" gracePeriod=25
Oct 30 19:14:38.423171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.414145    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:38.565150 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.565090    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:14:38.600721 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.600484    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:38.606164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.605282    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-maas-api-tls\") pod \"50dfb53c-b7bf-43eb-8956-10836baafa25\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") "
Oct 30 19:14:38.606164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.605412    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50dfb53c-b7bf-43eb-8956-10836baafa25-kube-api-access-48nrg\" (UniqueName: \"kubernetes.io/projected/50dfb53c-b7bf-43eb-8956-10836baafa25-kube-api-access-48nrg\") pod \"50dfb53c-b7bf-43eb-8956-10836baafa25\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") "
Oct 30 19:14:38.606164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.605609    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50dfb53c-b7bf-43eb-8956-10836baafa25-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50dfb53c-b7bf-43eb-8956-10836baafa25-openshift-service-ca\") pod \"50dfb53c-b7bf-43eb-8956-10836baafa25\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") "
Oct 30 19:14:38.606164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.605671    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-metrics-certs\" (UniqueName: \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-metrics-certs\") pod \"50dfb53c-b7bf-43eb-8956-10836baafa25\" (UID: \"50dfb53c-b7bf-43eb-8956-10836baafa25\") "
Oct 30 19:14:38.612784 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.612752    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50dfb53c-b7bf-43eb-8956-10836baafa25-openshift-service-ca" pod "50dfb53c-b7bf-43eb-8956-10836baafa25" (UID: "50dfb53c-b7bf-43eb-8956-10836baafa25"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:38.625000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.624424    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-metrics-certs" pod "50dfb53c-b7bf-43eb-8956-10836baafa25" (UID: "50dfb53c-b7bf-43eb-8956-10836baafa25"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:14:38.626519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.626493    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"]
Oct 30 19:14:38.628277 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.628254    4925 cpu_manager.go:414] "containerMap: removing container" podUID="50dfb53c-b7bf-43eb-8956-10836baafa25" containerName="maas-api"
Oct 30 19:14:38.628419 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.628404    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50dfb53c-b7bf-43eb-8956-10836baafa25" containerName="maas-api"
Oct 30 19:14:38.628840 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.628821    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="50dfb53c-b7bf-43eb-8956-10836baafa25" containerName="maas-api"
Oct 30 19:14:38.633195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.633156    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-maas-api-tls" pod "50dfb53c-b7bf-43eb-8956-10836baafa25" (UID: "50dfb53c-b7bf-43eb-8956-10836baafa25"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:14:38.635652 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.635634    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.635874 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.635735    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"]
Oct 30 19:14:38.637249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.637221    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50dfb53c-b7bf-43eb-8956-10836baafa25-kube-api-access-48nrg" pod "50dfb53c-b7bf-43eb-8956-10836baafa25" (UID: "50dfb53c-b7bf-43eb-8956-10836baafa25"). InnerVolumeSpecName "kube-api-access-48nrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:38.710337 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.710293    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-credential-socket\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.710561 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.710454    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7495850d-1b3d-4df3-8424-ebe372d9eb88-istiod-ca-cert\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.710561 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.710492    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-certs\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.710561 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.710558    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-envoy\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.710747 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.710591    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-token\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.710747 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.710651    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-data\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.710747 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.710740    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct72d\" (UniqueName: \"kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-kube-api-access-ct72d\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.710909 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.710772    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-socket\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.710909 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.710859    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-podinfo\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.711042 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.711025    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48nrg\" (UniqueName: \"kubernetes.io/projected/50dfb53c-b7bf-43eb-8956-10836baafa25-kube-api-access-48nrg\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:38.711158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.711047    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50dfb53c-b7bf-43eb-8956-10836baafa25-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:38.711158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.711064    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:38.711158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.711077    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50dfb53c-b7bf-43eb-8956-10836baafa25-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:38.812672 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.812637    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-credential-socket\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.812864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.812756    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7495850d-1b3d-4df3-8424-ebe372d9eb88-istiod-ca-cert\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.812864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.812780    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-certs\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.812864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.812842    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-envoy\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.813051 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.812877    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-token\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.813051 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.812946    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-data\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.813244 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.813079    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct72d\" (UniqueName: \"kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-kube-api-access-ct72d\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.813244 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.813129    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-socket\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.813244 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.813084    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-credential-socket\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.813426 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.813264    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-podinfo\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.813426 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.813364    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-data\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.813426 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.813415    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-socket\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.813607 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.813454    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-certs\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.813949 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.813926    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7495850d-1b3d-4df3-8424-ebe372d9eb88-istiod-ca-cert\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.815522 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.815501    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-envoy\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.815897 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.815877    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-podinfo\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.821020 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.820995    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-token\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.821375 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.821357    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct72d\" (UniqueName: \"kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-kube-api-access-ct72d\") pod \"e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") " pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:38.983944 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:38.983850    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:39.254812 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:39.253294    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:39.272024 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:39.271985    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg" event={"ID":"50dfb53c-b7bf-43eb-8956-10836baafa25","Type":"ContainerDied","Data":"df5171fb49c9e65aef7a4d283b16f6f925d253a98f70d3c6ec31f497539e9262"}
Oct 30 19:14:39.272252 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:39.272086    4925 scope.go:122] "RemoveContainer" containerID="e24dad0f0607e1ba5266f197af29c602995e7368dff97db72fbf0693138cee3d"
Oct 30 19:14:39.272317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:39.272301    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"
Oct 30 19:14:39.294502 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:39.294473    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"]
Oct 30 19:14:39.299190 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:39.299085    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:14:39.299317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:39.299245    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:14:39.299390 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:39.299330    4925 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:14:39.323192 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:39.323166    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"]
Oct 30 19:14:39.327799 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:39.327775    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-1dd09d-586678f4f9-nljkg"]
Oct 30 19:14:39.894120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:39.893935    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:40.306017 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:40.305984    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" event={"ID":"7495850d-1b3d-4df3-8424-ebe372d9eb88","Type":"ContainerStarted","Data":"2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697"}
Oct 30 19:14:40.306634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:40.306038    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" event={"ID":"7495850d-1b3d-4df3-8424-ebe372d9eb88","Type":"ContainerStarted","Data":"3c005a400bf205749057bda21abb0d5386d52450f40ad37fdd3338ad6aee9ff0"}
Oct 30 19:14:40.334830 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:40.334778    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" podStartSLOduration=9.334764304 podStartE2EDuration="9.334764304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:40.329194842 +0000 UTC m=+2302.045539922" watchObservedRunningTime="2026-10-30 19:14:40.334764304 +0000 UTC m=+2302.051109380"
Oct 30 19:14:40.426807 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:40.426774    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50dfb53c-b7bf-43eb-8956-10836baafa25" path="/var/lib/kubelet/pods/50dfb53c-b7bf-43eb-8956-10836baafa25/volumes"
Oct 30 19:14:40.730122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:40.721553    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:40.987170 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:40.986805    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:40.987170 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:40.986943    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused" start-of-body=
Oct 30 19:14:40.987170 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:40.986984    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" podUID="7495850d-1b3d-4df3-8424-ebe372d9eb88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused"
Oct 30 19:14:41.721831 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:41.721784    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:41.932035 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:41.932003    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:41.984258 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:41.984177    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused" start-of-body=
Oct 30 19:14:41.984258 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:41.984227    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" podUID="7495850d-1b3d-4df3-8424-ebe372d9eb88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused"
Oct 30 19:14:42.257574 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:42.257480    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:42.701121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:42.698133    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m" podUID="13c22a74-d32d-4400-bde2-eb77c344c8b9" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:14:42.790360 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:42.790311    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb" podUID="cfe122bd-8ed6-4a29-8931-8b93ef12f26b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:14:42.802472 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:42.795531    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"]
Oct 30 19:14:42.802946 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:42.802916    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy" containerID="cri-o://ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0" gracePeriod=30
Oct 30 19:14:42.988238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:42.988144    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused" start-of-body=
Oct 30 19:14:42.988525 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:42.988478    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" podUID="7495850d-1b3d-4df3-8424-ebe372d9eb88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused"
Oct 30 19:14:43.448138 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:43.447754    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" probeResult="failure" output="Get \"https://10.132.0.79:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:14:43.507407 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:14:43.507338    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:14:43.796213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:43.782305    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:43.985070 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:43.985034    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused" start-of-body=
Oct 30 19:14:43.985393 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:43.985364    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" podUID="7495850d-1b3d-4df3-8424-ebe372d9eb88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused"
Oct 30 19:14:44.739231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:44.739192    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:44.920350 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:44.920299    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"]
Oct 30 19:14:44.921595 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:44.921553    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerName="istio-proxy" containerID="cri-o://7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5" gracePeriod=30
Oct 30 19:14:44.947512 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:44.947467    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:45.058178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:45.054533    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused" start-of-body=
Oct 30 19:14:45.058178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:45.054615    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" podUID="7495850d-1b3d-4df3-8424-ebe372d9eb88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused"
Oct 30 19:14:45.240120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:45.233944    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:45.992118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:45.984516    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused" start-of-body=
Oct 30 19:14:45.992118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:45.984586    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" podUID="7495850d-1b3d-4df3-8424-ebe372d9eb88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused"
Oct 30 19:14:46.010280 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:46.010246    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:46.987588 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:46.987550    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused" start-of-body=
Oct 30 19:14:47.006275 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:47.006233    4925 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" podUID="7495850d-1b3d-4df3-8424-ebe372d9eb88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.238:15021/healthz/ready\": dial tcp 10.132.0.238:15021: connect: connection refused"
Oct 30 19:14:47.492794 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:47.492760    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:47.990977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:47.990950    4925 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:47.991507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:47.991483    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:47.992319 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:47.992293    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:14:48.238233 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.238207    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:14:48.323412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.322992    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-certs\") pod \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") "
Oct 30 19:14:48.323412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.323166    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-token\" (UniqueName: \"kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-token\") pod \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") "
Oct 30 19:14:48.323412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.323225    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istiod-ca-cert\") pod \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") "
Oct 30 19:14:48.323412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.323299    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-kube-api-access-flh6p\" (UniqueName: \"kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-kube-api-access-flh6p\") pod \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") "
Oct 30 19:14:48.323412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.323394    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-socket\") pod \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") "
Oct 30 19:14:48.323967 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.323518    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-credential-socket\") pod \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") "
Oct 30 19:14:48.323967 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.323575    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-envoy\") pod \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") "
Oct 30 19:14:48.323967 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.323604    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-data\") pod \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") "
Oct 30 19:14:48.323967 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.323646    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-podinfo\") pod \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\" (UID: \"b2d9c9bd-494d-42a7-98fc-bb1f11be4762\") "
Oct 30 19:14:48.325512 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.325479    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-certs" pod "b2d9c9bd-494d-42a7-98fc-bb1f11be4762" (UID: "b2d9c9bd-494d-42a7-98fc-bb1f11be4762"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:48.325753 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.325729    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-credential-socket" pod "b2d9c9bd-494d-42a7-98fc-bb1f11be4762" (UID: "b2d9c9bd-494d-42a7-98fc-bb1f11be4762"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:48.325937 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.325916    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-socket" pod "b2d9c9bd-494d-42a7-98fc-bb1f11be4762" (UID: "b2d9c9bd-494d-42a7-98fc-bb1f11be4762"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:48.332296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.326956    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-data" pod "b2d9c9bd-494d-42a7-98fc-bb1f11be4762" (UID: "b2d9c9bd-494d-42a7-98fc-bb1f11be4762"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:48.332296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.327896    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:48.332296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.330000    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-envoy" pod "b2d9c9bd-494d-42a7-98fc-bb1f11be4762" (UID: "b2d9c9bd-494d-42a7-98fc-bb1f11be4762"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:48.332296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.332182    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-podinfo" pod "b2d9c9bd-494d-42a7-98fc-bb1f11be4762" (UID: "b2d9c9bd-494d-42a7-98fc-bb1f11be4762"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:14:48.379569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.366307    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istiod-ca-cert" pod "b2d9c9bd-494d-42a7-98fc-bb1f11be4762" (UID: "b2d9c9bd-494d-42a7-98fc-bb1f11be4762"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:48.424131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.410937    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-kube-api-access-flh6p" pod "b2d9c9bd-494d-42a7-98fc-bb1f11be4762" (UID: "b2d9c9bd-494d-42a7-98fc-bb1f11be4762"). InnerVolumeSpecName "kube-api-access-flh6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:48.424131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.413215    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-token" pod "b2d9c9bd-494d-42a7-98fc-bb1f11be4762" (UID: "b2d9c9bd-494d-42a7-98fc-bb1f11be4762"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:48.429378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.429338    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flh6p\" (UniqueName: \"kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-kube-api-access-flh6p\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:48.429598 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.429579    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:48.429727 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.429714    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:48.429836 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.429825    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:48.429942 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.429931    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:48.430049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.430038    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:48.430165 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.430154    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:48.430278 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.430267    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:48.430381 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.430370    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d9c9bd-494d-42a7-98fc-bb1f11be4762-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:48.542858 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.542825    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"]
Oct 30 19:14:48.550386 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.550358    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"]
Oct 30 19:14:48.696194 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.696078    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:48.915268 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.915238    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:48.918019 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.917274    4925 generic.go:356] "Generic (PLEG): container finished" podID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerID="ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0" exitCode=0
Oct 30 19:14:48.921336 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.919123    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"
Oct 30 19:14:48.921336 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.919826    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" event={"ID":"b2d9c9bd-494d-42a7-98fc-bb1f11be4762","Type":"ContainerDied","Data":"ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0"}
Oct 30 19:14:48.921336 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.919877    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh" event={"ID":"b2d9c9bd-494d-42a7-98fc-bb1f11be4762","Type":"ContainerDied","Data":"ba4980ba22bc4ff584858f09442fed8585d56d960c980aeb30a9aaaff63f4359"}
Oct 30 19:14:48.921336 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.919909    4925 scope.go:122] "RemoveContainer" containerID="ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0"
Oct 30 19:14:48.994904 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:48.994811    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"]
Oct 30 19:14:49.027206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:49.027155    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-ecc266-openshift-default-68bdc475d9-cdvjh"]
Oct 30 19:14:49.074119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:49.066175    4925 scope.go:122] "RemoveContainer" containerID="dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65"
Oct 30 19:14:49.156125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:49.148071    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:49.187746 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:49.175756    4925 scope.go:122] "RemoveContainer" containerID="ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0"
Oct 30 19:14:49.196123 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:14:49.194269    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0\": container with ID starting with ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0 not found: ID does not exist" containerID="ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0"
Oct 30 19:14:49.196123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:49.194318    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0"} err="failed to get container status \"ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0\": rpc error: code = NotFound desc = could not find container \"ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0\": container with ID starting with ff81a5c6e9d95e315001186cb14ed6a1477204f639d8bcada1e9e1626be57ad0 not found: ID does not exist"
Oct 30 19:14:49.196123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:49.194345    4925 scope.go:122] "RemoveContainer" containerID="dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65"
Oct 30 19:14:49.196123 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:14:49.194739    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65\": container with ID starting with dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65 not found: ID does not exist" containerID="dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65"
Oct 30 19:14:49.196123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:49.194768    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65"} err="failed to get container status \"dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65\": rpc error: code = NotFound desc = could not find container \"dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65\": container with ID starting with dd2be3a674c46360a35dc351d57ef00dd4d991feab7424cfbeb82c73d600cf65 not found: ID does not exist"
Oct 30 19:14:49.972273 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:49.972234    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:50.246842 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.246778    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:14:50.334493 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.334455    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-data\") pod \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") "
Oct 30 19:14:50.334754 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.334538    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-socket\") pod \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") "
Oct 30 19:14:50.334859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.334822    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-credential-socket\") pod \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") "
Oct 30 19:14:50.334953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.334920    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-podinfo\") pod \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") "
Oct 30 19:14:50.335067 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.334999    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-envoy\") pod \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") "
Oct 30 19:14:50.335195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.335121    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-token\" (UniqueName: \"kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-token\") pod \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") "
Oct 30 19:14:50.335195 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.335169    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-certs\") pod \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") "
Oct 30 19:14:50.335411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.335243    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istiod-ca-cert\") pod \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") "
Oct 30 19:14:50.335411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.335272    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-kube-api-access-t7hv8\" (UniqueName: \"kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-kube-api-access-t7hv8\") pod \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\" (UID: \"f78613e0-ea60-4e84-b6f9-e61564d1cdcc\") "
Oct 30 19:14:50.336225 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.335919    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istiod-ca-cert" pod "f78613e0-ea60-4e84-b6f9-e61564d1cdcc" (UID: "f78613e0-ea60-4e84-b6f9-e61564d1cdcc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:50.336225 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.336014    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-data" pod "f78613e0-ea60-4e84-b6f9-e61564d1cdcc" (UID: "f78613e0-ea60-4e84-b6f9-e61564d1cdcc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:50.336401 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.336350    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-certs" pod "f78613e0-ea60-4e84-b6f9-e61564d1cdcc" (UID: "f78613e0-ea60-4e84-b6f9-e61564d1cdcc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:50.336594 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.336504    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:50.336594 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.336522    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:50.336924 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.336815    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-credential-socket" pod "f78613e0-ea60-4e84-b6f9-e61564d1cdcc" (UID: "f78613e0-ea60-4e84-b6f9-e61564d1cdcc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:50.337039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.337016    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-socket" pod "f78613e0-ea60-4e84-b6f9-e61564d1cdcc" (UID: "f78613e0-ea60-4e84-b6f9-e61564d1cdcc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:50.341817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.340360    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-envoy" pod "f78613e0-ea60-4e84-b6f9-e61564d1cdcc" (UID: "f78613e0-ea60-4e84-b6f9-e61564d1cdcc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:50.342054 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.342029    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-podinfo" pod "f78613e0-ea60-4e84-b6f9-e61564d1cdcc" (UID: "f78613e0-ea60-4e84-b6f9-e61564d1cdcc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:14:50.342388 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.342366    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-token" pod "f78613e0-ea60-4e84-b6f9-e61564d1cdcc" (UID: "f78613e0-ea60-4e84-b6f9-e61564d1cdcc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:50.342724 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.342704    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-kube-api-access-t7hv8" pod "f78613e0-ea60-4e84-b6f9-e61564d1cdcc" (UID: "f78613e0-ea60-4e84-b6f9-e61564d1cdcc"). InnerVolumeSpecName "kube-api-access-t7hv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:50.430824 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.430786    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" path="/var/lib/kubelet/pods/b2d9c9bd-494d-42a7-98fc-bb1f11be4762/volumes"
Oct 30 19:14:50.437454 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.437424    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:50.437566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.437459    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:50.437566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.437470    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:50.437566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.437480    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:50.437566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.437489    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:50.437566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.437500    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7hv8\" (UniqueName: \"kubernetes.io/projected/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-kube-api-access-t7hv8\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:50.437566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.437509    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78613e0-ea60-4e84-b6f9-e61564d1cdcc-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:50.766170 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:50.766136    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:51.001973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:51.001935    4925 generic.go:356] "Generic (PLEG): container finished" podID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerID="7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5" exitCode=0
Oct 30 19:14:51.002554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:51.002005    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"
Oct 30 19:14:51.002554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:51.002017    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" event={"ID":"f78613e0-ea60-4e84-b6f9-e61564d1cdcc","Type":"ContainerDied","Data":"7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5"}
Oct 30 19:14:51.002554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:51.002165    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp" event={"ID":"f78613e0-ea60-4e84-b6f9-e61564d1cdcc","Type":"ContainerDied","Data":"14b9b73e31483ec6fcd82a1b7e316434594134b0cb0e089d506e4990cffe18ae"}
Oct 30 19:14:51.002554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:51.002210    4925 scope.go:122] "RemoveContainer" containerID="7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5"
Oct 30 19:14:51.016566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:51.016549    4925 scope.go:122] "RemoveContainer" containerID="7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5"
Oct 30 19:14:51.016908 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:14:51.016881    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5\": container with ID starting with 7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5 not found: ID does not exist" containerID="7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5"
Oct 30 19:14:51.016976 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:51.016922    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5"} err="failed to get container status \"7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5\": rpc error: code = NotFound desc = could not find container \"7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5\": container with ID starting with 7591e749d0fe49033a0c915a6d199543634e6c0d6b6f160b27e9f11e248acbd5 not found: ID does not exist"
Oct 30 19:14:51.027242 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:51.027200    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"]
Oct 30 19:14:51.033367 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:51.033343    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-1dd09d-openshift-default-7477c8cdbf-cqzrp"]
Oct 30 19:14:51.369176 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:51.367981    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:52.092582 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:52.092332    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:52.394391 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:52.394295    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:52.426425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:52.426394    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" path="/var/lib/kubelet/pods/f78613e0-ea60-4e84-b6f9-e61564d1cdcc/volumes"
Oct 30 19:14:52.458460 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:52.458421    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb" podUID="cfe122bd-8ed6-4a29-8931-8b93ef12f26b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:14:52.458460 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:52.458444    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m" podUID="13c22a74-d32d-4400-bde2-eb77c344c8b9" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:14:52.573860 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:52.573816    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:52.783749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:52.782564    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:53.546973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:53.546904    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2" podUID="822c68de-9860-4a2d-b7ae-145bd319f117" containerName="payload-processing" containerID="cri-o://8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8" gracePeriod=25
Oct 30 19:14:53.552318 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:53.552165    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf" podUID="1aeb450f-292b-4274-8bf4-2e8b6e686d50" containerName="payload-pre-processing" containerID="cri-o://bae428d9a99cff9cc2a085b7b2ab681152e6d4c61d95b279f77e6b1dfa6f80cb" gracePeriod=25
Oct 30 19:14:53.563962 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:53.563931    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:53.624171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:53.623284    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"]
Oct 30 19:14:53.763851 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:53.763817    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:53.990122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:53.985792    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:54.485118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:54.478263    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:55.113405 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:55.104161    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:55.570725 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:55.570686    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"]
Oct 30 19:14:55.725577 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:55.725530    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:55.916591 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:55.916556    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:56.566702 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:56.566664    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:57.260114 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:57.258850    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:58.036141 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.032499    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:58.525596 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.525542    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"
Oct 30 19:14:58.578645 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.578613    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-plugins-config-volume\") pod \"cfe122bd-8ed6-4a29-8931-8b93ef12f26b\" (UID: \"cfe122bd-8ed6-4a29-8931-8b93ef12f26b\") "
Oct 30 19:14:58.579077 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.579051    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-plugins-config-volume" pod "cfe122bd-8ed6-4a29-8931-8b93ef12f26b" (UID: "cfe122bd-8ed6-4a29-8931-8b93ef12f26b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:58.581353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.581328    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-kube-api-access-nj48m\" (UniqueName: \"kubernetes.io/projected/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-kube-api-access-nj48m\") pod \"cfe122bd-8ed6-4a29-8931-8b93ef12f26b\" (UID: \"cfe122bd-8ed6-4a29-8931-8b93ef12f26b\") "
Oct 30 19:14:58.594349 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.587324    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-kube-api-access-nj48m" pod "cfe122bd-8ed6-4a29-8931-8b93ef12f26b" (UID: "cfe122bd-8ed6-4a29-8931-8b93ef12f26b"). InnerVolumeSpecName "kube-api-access-nj48m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:58.624353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.624323    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq" podUID="d8afa108-8874-4821-9a23-784ff39e42ce" containerName="payload-pre-processing" containerID="cri-o://33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc" gracePeriod=25
Oct 30 19:14:58.703047 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.697129    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:58.714180 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.713761    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:58.714180 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.713798    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj48m\" (UniqueName: \"kubernetes.io/projected/cfe122bd-8ed6-4a29-8931-8b93ef12f26b-kube-api-access-nj48m\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:58.769760 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.769386    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"
Oct 30 19:14:58.815487 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.815396    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/13c22a74-d32d-4400-bde2-eb77c344c8b9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13c22a74-d32d-4400-bde2-eb77c344c8b9-plugins-config-volume\") pod \"13c22a74-d32d-4400-bde2-eb77c344c8b9\" (UID: \"13c22a74-d32d-4400-bde2-eb77c344c8b9\") "
Oct 30 19:14:58.815487 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.815454    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13c22a74-d32d-4400-bde2-eb77c344c8b9-kube-api-access-ks9kj\" (UniqueName: \"kubernetes.io/projected/13c22a74-d32d-4400-bde2-eb77c344c8b9-kube-api-access-ks9kj\") pod \"13c22a74-d32d-4400-bde2-eb77c344c8b9\" (UID: \"13c22a74-d32d-4400-bde2-eb77c344c8b9\") "
Oct 30 19:14:58.816892 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.816859    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13c22a74-d32d-4400-bde2-eb77c344c8b9-plugins-config-volume" pod "13c22a74-d32d-4400-bde2-eb77c344c8b9" (UID: "13c22a74-d32d-4400-bde2-eb77c344c8b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:58.820108 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.819654    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13c22a74-d32d-4400-bde2-eb77c344c8b9-kube-api-access-ks9kj" pod "13c22a74-d32d-4400-bde2-eb77c344c8b9" (UID: "13c22a74-d32d-4400-bde2-eb77c344c8b9"). InnerVolumeSpecName "kube-api-access-ks9kj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:58.918172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.918042    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13c22a74-d32d-4400-bde2-eb77c344c8b9-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:58.918172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.918072    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ks9kj\" (UniqueName: \"kubernetes.io/projected/13c22a74-d32d-4400-bde2-eb77c344c8b9-kube-api-access-ks9kj\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:58.952298 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:58.952266    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"
Oct 30 19:14:59.019216 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.019188    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8afa108-8874-4821-9a23-784ff39e42ce-kube-api-access-pmzd7\" (UniqueName: \"kubernetes.io/projected/d8afa108-8874-4821-9a23-784ff39e42ce-kube-api-access-pmzd7\") pod \"d8afa108-8874-4821-9a23-784ff39e42ce\" (UID: \"d8afa108-8874-4821-9a23-784ff39e42ce\") "
Oct 30 19:14:59.019434 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.019418    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8afa108-8874-4821-9a23-784ff39e42ce-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8afa108-8874-4821-9a23-784ff39e42ce-plugins-config-volume\") pod \"d8afa108-8874-4821-9a23-784ff39e42ce\" (UID: \"d8afa108-8874-4821-9a23-784ff39e42ce\") "
Oct 30 19:14:59.019878 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.019848    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8afa108-8874-4821-9a23-784ff39e42ce-plugins-config-volume" pod "d8afa108-8874-4821-9a23-784ff39e42ce" (UID: "d8afa108-8874-4821-9a23-784ff39e42ce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:59.020215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.020195    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8afa108-8874-4821-9a23-784ff39e42ce-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:59.021543 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.021516    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8afa108-8874-4821-9a23-784ff39e42ce-kube-api-access-pmzd7" pod "d8afa108-8874-4821-9a23-784ff39e42ce" (UID: "d8afa108-8874-4821-9a23-784ff39e42ce"). InnerVolumeSpecName "kube-api-access-pmzd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:59.107739 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.107660    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"]
Oct 30 19:14:59.110526 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.107932    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq" podUID="5bb2e79e-a70d-48c3-b8f5-622717c110c3" containerName="maas-api" containerID="cri-o://1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58" gracePeriod=30
Oct 30 19:14:59.123274 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.123243    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmzd7\" (UniqueName: \"kubernetes.io/projected/d8afa108-8874-4821-9a23-784ff39e42ce-kube-api-access-pmzd7\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:59.197352 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.197316    4925 generic.go:356] "Generic (PLEG): container finished" podID="13c22a74-d32d-4400-bde2-eb77c344c8b9" containerID="7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2" exitCode=0
Oct 30 19:14:59.197552 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.197366    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m" event={"ID":"13c22a74-d32d-4400-bde2-eb77c344c8b9","Type":"ContainerDied","Data":"7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2"}
Oct 30 19:14:59.197552 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.197397    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"
Oct 30 19:14:59.197552 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.197434    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m" event={"ID":"13c22a74-d32d-4400-bde2-eb77c344c8b9","Type":"ContainerDied","Data":"3836e3c1dcfb65d1b01e2bb1f52f42f00b04e8c1e27835873a7adce58c617a85"}
Oct 30 19:14:59.197552 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.197463    4925 scope.go:122] "RemoveContainer" containerID="7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2"
Oct 30 19:14:59.199252 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.199225    4925 generic.go:356] "Generic (PLEG): container finished" podID="d8afa108-8874-4821-9a23-784ff39e42ce" containerID="33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc" exitCode=0
Oct 30 19:14:59.199362 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.199265    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"
Oct 30 19:14:59.199409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.199347    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq" event={"ID":"d8afa108-8874-4821-9a23-784ff39e42ce","Type":"ContainerDied","Data":"33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc"}
Oct 30 19:14:59.199409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.199402    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq" event={"ID":"d8afa108-8874-4821-9a23-784ff39e42ce","Type":"ContainerDied","Data":"15f2689fe8c3da1e5bcde2e9813107101f6c4bc45634b2d7f035969dba4c74ac"}
Oct 30 19:14:59.201253 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.201229    4925 generic.go:356] "Generic (PLEG): container finished" podID="cfe122bd-8ed6-4a29-8931-8b93ef12f26b" containerID="1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596" exitCode=0
Oct 30 19:14:59.201411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.201324    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb" event={"ID":"cfe122bd-8ed6-4a29-8931-8b93ef12f26b","Type":"ContainerDied","Data":"1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596"}
Oct 30 19:14:59.201411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.201361    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb" event={"ID":"cfe122bd-8ed6-4a29-8931-8b93ef12f26b","Type":"ContainerDied","Data":"ab67541cce7a925b26bc775080c103c31acf1e166aa1dc77af19a85416efffc9"}
Oct 30 19:14:59.201528 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.201454    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"
Oct 30 19:14:59.233451 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.233404    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"]
Oct 30 19:14:59.237612 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.237588    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:59.241868 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.241846    4925 scope.go:122] "RemoveContainer" containerID="7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2"
Oct 30 19:14:59.242224 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:14:59.242200    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2\": container with ID starting with 7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2 not found: ID does not exist" containerID="7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2"
Oct 30 19:14:59.242344 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.242240    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2"} err="failed to get container status \"7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2\": rpc error: code = NotFound desc = could not find container \"7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2\": container with ID starting with 7c4e193d5cad2f979ac98c07a1cf43c369a431d9123acd5098841cb0cd51fdb2 not found: ID does not exist"
Oct 30 19:14:59.242344 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.242263    4925 scope.go:122] "RemoveContainer" containerID="33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc"
Oct 30 19:14:59.242975 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.242751    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-baa504-649698ff67-7xrxq"]
Oct 30 19:14:59.250577 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.250546    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"]
Oct 30 19:14:59.254339 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.254317    4925 scope.go:122] "RemoveContainer" containerID="33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc"
Oct 30 19:14:59.254680 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:14:59.254654    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc\": container with ID starting with 33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc not found: ID does not exist" containerID="33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc"
Oct 30 19:14:59.254777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.254690    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc"} err="failed to get container status \"33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc\": rpc error: code = NotFound desc = could not find container \"33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc\": container with ID starting with 33816af298eac8ee7e33b35db521a5b5732268fd997ca4eee308be201a024afc not found: ID does not exist"
Oct 30 19:14:59.254777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.254713    4925 scope.go:122] "RemoveContainer" containerID="1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596"
Oct 30 19:14:59.255693 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.255674    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-1dd09d-6fdfc749bd-w5q6m"]
Oct 30 19:14:59.263271 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.263247    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"]
Oct 30 19:14:59.275306 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.275283    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-1dd09d-6f8fd8d8ff-7fkdb"]
Oct 30 19:14:59.277424 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.277404    4925 scope.go:122] "RemoveContainer" containerID="1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596"
Oct 30 19:14:59.277834 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:14:59.277775    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596\": container with ID starting with 1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596 not found: ID does not exist" containerID="1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596"
Oct 30 19:14:59.277834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.277810    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596"} err="failed to get container status \"1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596\": rpc error: code = NotFound desc = could not find container \"1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596\": container with ID starting with 1a9ae85b595c53294f9f946983a2484ffd064286431c4370304f0d066883f596 not found: ID does not exist"
Oct 30 19:14:59.426424 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.426366    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:59.702198 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.694205    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:59.901382 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.901339    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:59.915338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.915309    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:14:59.918642 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.918606    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5bb2e79e-a70d-48c3-b8f5-622717c110c3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb2e79e-a70d-48c3-b8f5-622717c110c3-openshift-service-ca\") pod \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") "
Oct 30 19:14:59.918806 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.918706    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5bb2e79e-a70d-48c3-b8f5-622717c110c3-kube-api-access-6jwtc\" (UniqueName: \"kubernetes.io/projected/5bb2e79e-a70d-48c3-b8f5-622717c110c3-kube-api-access-6jwtc\") pod \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") "
Oct 30 19:14:59.918914 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.918895    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-metrics-certs\") pod \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") "
Oct 30 19:14:59.918991 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.918953    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-maas-api-tls\") pod \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\" (UID: \"5bb2e79e-a70d-48c3-b8f5-622717c110c3\") "
Oct 30 19:14:59.919053 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.919024    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bb2e79e-a70d-48c3-b8f5-622717c110c3-openshift-service-ca" pod "5bb2e79e-a70d-48c3-b8f5-622717c110c3" (UID: "5bb2e79e-a70d-48c3-b8f5-622717c110c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:59.919946 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.919920    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5bb2e79e-a70d-48c3-b8f5-622717c110c3-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:59.921623 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.921470    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5bb2e79e-a70d-48c3-b8f5-622717c110c3-kube-api-access-6jwtc" pod "5bb2e79e-a70d-48c3-b8f5-622717c110c3" (UID: "5bb2e79e-a70d-48c3-b8f5-622717c110c3"). InnerVolumeSpecName "kube-api-access-6jwtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:59.921623 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.921580    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-metrics-certs" pod "5bb2e79e-a70d-48c3-b8f5-622717c110c3" (UID: "5bb2e79e-a70d-48c3-b8f5-622717c110c3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:14:59.922010 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:14:59.921992    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-maas-api-tls" pod "5bb2e79e-a70d-48c3-b8f5-622717c110c3" (UID: "5bb2e79e-a70d-48c3-b8f5-622717c110c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:00.022162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.022024    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jwtc\" (UniqueName: \"kubernetes.io/projected/5bb2e79e-a70d-48c3-b8f5-622717c110c3-kube-api-access-6jwtc\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:00.022162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.022065    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:00.022162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.022079    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5bb2e79e-a70d-48c3-b8f5-622717c110c3-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:00.094908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.094873    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:00.217070 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.217037    4925 generic.go:356] "Generic (PLEG): container finished" podID="5bb2e79e-a70d-48c3-b8f5-622717c110c3" containerID="1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58" exitCode=0
Oct 30 19:15:00.217583 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.217230    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq" event={"ID":"5bb2e79e-a70d-48c3-b8f5-622717c110c3","Type":"ContainerDied","Data":"1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58"}
Oct 30 19:15:00.217583 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.217333    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq" event={"ID":"5bb2e79e-a70d-48c3-b8f5-622717c110c3","Type":"ContainerDied","Data":"ae46976ff78957a738053af221db9326ba7dd81a094323b33263e9241a6da814"}
Oct 30 19:15:00.217583 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.217395    4925 scope.go:122] "RemoveContainer" containerID="1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58"
Oct 30 19:15:00.217583 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.217495    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"
Oct 30 19:15:00.234167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.234148    4925 scope.go:122] "RemoveContainer" containerID="1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58"
Oct 30 19:15:00.234475 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:00.234445    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58\": container with ID starting with 1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58 not found: ID does not exist" containerID="1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58"
Oct 30 19:15:00.234567 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.234486    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58"} err="failed to get container status \"1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58\": rpc error: code = NotFound desc = could not find container \"1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58\": container with ID starting with 1708be20b5157604eee8f1f84a3954732e160346ed42eda01cc4b79cb5061c58 not found: ID does not exist"
Oct 30 19:15:00.235423 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.235400    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"]
Oct 30 19:15:00.236859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.236834    4925 cpu_manager.go:414] "containerMap: removing container" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy"
Oct 30 19:15:00.236859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.236859    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy"
Oct 30 19:15:00.237003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.236887    4925 cpu_manager.go:414] "containerMap: removing container" podUID="13c22a74-d32d-4400-bde2-eb77c344c8b9" containerName="payload-processing"
Oct 30 19:15:00.237003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.236895    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13c22a74-d32d-4400-bde2-eb77c344c8b9" containerName="payload-processing"
Oct 30 19:15:00.237003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.236922    4925 cpu_manager.go:414] "containerMap: removing container" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerName="istio-proxy"
Oct 30 19:15:00.237003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.236927    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerName="istio-proxy"
Oct 30 19:15:00.237003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.236949    4925 cpu_manager.go:414] "containerMap: removing container" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy"
Oct 30 19:15:00.237003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.236954    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy"
Oct 30 19:15:00.237003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.236974    4925 cpu_manager.go:414] "containerMap: removing container" podUID="5bb2e79e-a70d-48c3-b8f5-622717c110c3" containerName="maas-api"
Oct 30 19:15:00.237003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.236980    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bb2e79e-a70d-48c3-b8f5-622717c110c3" containerName="maas-api"
Oct 30 19:15:00.237003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.236998    4925 cpu_manager.go:414] "containerMap: removing container" podUID="cfe122bd-8ed6-4a29-8931-8b93ef12f26b" containerName="payload-pre-processing"
Oct 30 19:15:00.237003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.237002    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfe122bd-8ed6-4a29-8931-8b93ef12f26b" containerName="payload-pre-processing"
Oct 30 19:15:00.237519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.237012    4925 cpu_manager.go:414] "containerMap: removing container" podUID="d8afa108-8874-4821-9a23-784ff39e42ce" containerName="payload-pre-processing"
Oct 30 19:15:00.237519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.237018    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8afa108-8874-4821-9a23-784ff39e42ce" containerName="payload-pre-processing"
Oct 30 19:15:00.237519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.237194    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy"
Oct 30 19:15:00.237519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.237207    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="5bb2e79e-a70d-48c3-b8f5-622717c110c3" containerName="maas-api"
Oct 30 19:15:00.237519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.237216    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfe122bd-8ed6-4a29-8931-8b93ef12f26b" containerName="payload-pre-processing"
Oct 30 19:15:00.237519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.237224    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8afa108-8874-4821-9a23-784ff39e42ce" containerName="payload-pre-processing"
Oct 30 19:15:00.237519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.237232    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2d9c9bd-494d-42a7-98fc-bb1f11be4762" containerName="istio-proxy"
Oct 30 19:15:00.237519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.237240    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="13c22a74-d32d-4400-bde2-eb77c344c8b9" containerName="payload-processing"
Oct 30 19:15:00.237519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.237248    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="f78613e0-ea60-4e84-b6f9-e61564d1cdcc" containerName="istio-proxy"
Oct 30 19:15:00.243588 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.243560    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"
Oct 30 19:15:00.254271 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.254249    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-94hcn\""
Oct 30 19:15:00.272437 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.272372    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"]
Oct 30 19:15:00.293396 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.293369    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"]
Oct 30 19:15:00.303315 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.303274    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"]
Oct 30 19:15:00.311196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.311174    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"
Oct 30 19:15:00.312000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.311977    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"]
Oct 30 19:15:00.317700 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.317680    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-e51c10-6bfd6dc7f6-n8wxq"]
Oct 30 19:15:00.317843 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.317791    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"
Oct 30 19:15:00.322193 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.322172    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"]
Oct 30 19:15:00.327516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.326242    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"]
Oct 30 19:15:00.330156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.329857    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrqfk\" (UniqueName: \"kubernetes.io/projected/673b9cea-e53a-46be-968f-3ad97d7deb2d-kube-api-access-nrqfk\") pod \"maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd\" (UID: \"673b9cea-e53a-46be-968f-3ad97d7deb2d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"
Oct 30 19:15:00.330156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.329905    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4tg5\" (UniqueName: \"kubernetes.io/projected/50839431-baf0-4a66-a775-54be5ea96723-kube-api-access-g4tg5\") pod \"maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t\" (UID: \"50839431-baf0-4a66-a775-54be5ea96723\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"
Oct 30 19:15:00.330156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.329950    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/673b9cea-e53a-46be-968f-3ad97d7deb2d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd\" (UID: \"673b9cea-e53a-46be-968f-3ad97d7deb2d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"
Oct 30 19:15:00.330156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.330015    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f8aae7-be84-42e4-b3f3-66d95a9e5579-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf\" (UID: \"71f8aae7-be84-42e4-b3f3-66d95a9e5579\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"
Oct 30 19:15:00.330415 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.330281    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z277g\" (UniqueName: \"kubernetes.io/projected/71f8aae7-be84-42e4-b3f3-66d95a9e5579-kube-api-access-z277g\") pod \"maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf\" (UID: \"71f8aae7-be84-42e4-b3f3-66d95a9e5579\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"
Oct 30 19:15:00.330415 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.330407    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50839431-baf0-4a66-a775-54be5ea96723-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t\" (UID: \"50839431-baf0-4a66-a775-54be5ea96723\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"
Oct 30 19:15:00.429261 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.429228    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13c22a74-d32d-4400-bde2-eb77c344c8b9" path="/var/lib/kubelet/pods/13c22a74-d32d-4400-bde2-eb77c344c8b9/volumes"
Oct 30 19:15:00.429757 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.429735    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5bb2e79e-a70d-48c3-b8f5-622717c110c3" path="/var/lib/kubelet/pods/5bb2e79e-a70d-48c3-b8f5-622717c110c3/volumes"
Oct 30 19:15:00.430298 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.430278    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfe122bd-8ed6-4a29-8931-8b93ef12f26b" path="/var/lib/kubelet/pods/cfe122bd-8ed6-4a29-8931-8b93ef12f26b/volumes"
Oct 30 19:15:00.431063 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.431042    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8afa108-8874-4821-9a23-784ff39e42ce" path="/var/lib/kubelet/pods/d8afa108-8874-4821-9a23-784ff39e42ce/volumes"
Oct 30 19:15:00.431923 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.431890    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50839431-baf0-4a66-a775-54be5ea96723-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t\" (UID: \"50839431-baf0-4a66-a775-54be5ea96723\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"
Oct 30 19:15:00.432241 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.432218    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrqfk\" (UniqueName: \"kubernetes.io/projected/673b9cea-e53a-46be-968f-3ad97d7deb2d-kube-api-access-nrqfk\") pod \"maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd\" (UID: \"673b9cea-e53a-46be-968f-3ad97d7deb2d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"
Oct 30 19:15:00.432339 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.432277    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4tg5\" (UniqueName: \"kubernetes.io/projected/50839431-baf0-4a66-a775-54be5ea96723-kube-api-access-g4tg5\") pod \"maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t\" (UID: \"50839431-baf0-4a66-a775-54be5ea96723\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"
Oct 30 19:15:00.432418 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.432338    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/673b9cea-e53a-46be-968f-3ad97d7deb2d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd\" (UID: \"673b9cea-e53a-46be-968f-3ad97d7deb2d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"
Oct 30 19:15:00.432418 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.432374    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f8aae7-be84-42e4-b3f3-66d95a9e5579-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf\" (UID: \"71f8aae7-be84-42e4-b3f3-66d95a9e5579\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"
Oct 30 19:15:00.432691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.432665    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z277g\" (UniqueName: \"kubernetes.io/projected/71f8aae7-be84-42e4-b3f3-66d95a9e5579-kube-api-access-z277g\") pod \"maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf\" (UID: \"71f8aae7-be84-42e4-b3f3-66d95a9e5579\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"
Oct 30 19:15:00.432691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.432681    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50839431-baf0-4a66-a775-54be5ea96723-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t\" (UID: \"50839431-baf0-4a66-a775-54be5ea96723\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"
Oct 30 19:15:00.433475 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.433446    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/673b9cea-e53a-46be-968f-3ad97d7deb2d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd\" (UID: \"673b9cea-e53a-46be-968f-3ad97d7deb2d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"
Oct 30 19:15:00.433861 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.433840    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f8aae7-be84-42e4-b3f3-66d95a9e5579-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf\" (UID: \"71f8aae7-be84-42e4-b3f3-66d95a9e5579\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"
Oct 30 19:15:00.448549 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.448520    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4tg5\" (UniqueName: \"kubernetes.io/projected/50839431-baf0-4a66-a775-54be5ea96723-kube-api-access-g4tg5\") pod \"maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t\" (UID: \"50839431-baf0-4a66-a775-54be5ea96723\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"
Oct 30 19:15:00.448708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.448691    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z277g\" (UniqueName: \"kubernetes.io/projected/71f8aae7-be84-42e4-b3f3-66d95a9e5579-kube-api-access-z277g\") pod \"maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf\" (UID: \"71f8aae7-be84-42e4-b3f3-66d95a9e5579\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"
Oct 30 19:15:00.449231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.449208    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrqfk\" (UniqueName: \"kubernetes.io/projected/673b9cea-e53a-46be-968f-3ad97d7deb2d-kube-api-access-nrqfk\") pod \"maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd\" (UID: \"673b9cea-e53a-46be-968f-3ad97d7deb2d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"
Oct 30 19:15:00.568986 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.568907    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"
Oct 30 19:15:00.571996 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.571968    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd" podUID="547ccaea-7848-46dd-a638-3411f31835f7" containerName="payload-processing" containerID="cri-o://158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae" gracePeriod=25
Oct 30 19:15:00.643425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.643397    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"
Oct 30 19:15:00.650583 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.650378    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"
Oct 30 19:15:00.930236 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.929877    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:00.954445 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:15:00.950437    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f8aae7_be84_42e4_b3f3_66d95a9e5579.slice/crio-3f640ac526adeea8c1589de320c3b11e63cad2dd9f2e739813cf8052f28f3f3a WatchSource:0}: Error finding container 3f640ac526adeea8c1589de320c3b11e63cad2dd9f2e739813cf8052f28f3f3a: Status 404 returned error can't find the container with id 3f640ac526adeea8c1589de320c3b11e63cad2dd9f2e739813cf8052f28f3f3a
Oct 30 19:15:00.963400 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.963367    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"]
Oct 30 19:15:00.983300 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.983270    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"]
Oct 30 19:15:00.992642 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:15:00.992603    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50839431_baf0_4a66_a775_54be5ea96723.slice/crio-c17af425cbba80882012d2f67022438585ab3733982734678cd6cd3c66e6fa5a WatchSource:0}: Error finding container c17af425cbba80882012d2f67022438585ab3733982734678cd6cd3c66e6fa5a: Status 404 returned error can't find the container with id c17af425cbba80882012d2f67022438585ab3733982734678cd6cd3c66e6fa5a
Oct 30 19:15:00.993268 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:00.993169    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"]
Oct 30 19:15:01.229703 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:01.229668    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t" event={"ID":"50839431-baf0-4a66-a775-54be5ea96723","Type":"ContainerStarted","Data":"c17af425cbba80882012d2f67022438585ab3733982734678cd6cd3c66e6fa5a"}
Oct 30 19:15:01.231139 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:01.231078    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd" event={"ID":"673b9cea-e53a-46be-968f-3ad97d7deb2d","Type":"ContainerStarted","Data":"d44ea92c6ea3e9bcb845866c1da78f174e2f990e704b166bdeaaacaa7d48275c"}
Oct 30 19:15:01.232525 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:01.232497    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf" event={"ID":"71f8aae7-be84-42e4-b3f3-66d95a9e5579","Type":"ContainerStarted","Data":"3f640ac526adeea8c1589de320c3b11e63cad2dd9f2e739813cf8052f28f3f3a"}
Oct 30 19:15:01.783433 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:01.779696    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:02.018328 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.018287    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2" podUID="822c68de-9860-4a2d-b7ae-145bd319f117" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:02.018493 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.018424    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf" podUID="1aeb450f-292b-4274-8bf4-2e8b6e686d50" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:02.244800 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.244706    4925 generic.go:356] "Generic (PLEG): container finished" podID="50839431-baf0-4a66-a775-54be5ea96723" containerID="8d98d857f7904b0b935249cb177d6416a9a0c36465b5f2850b59e86cbd2ae02d" exitCode=60
Oct 30 19:15:02.244800 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.244784    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t" event={"ID":"50839431-baf0-4a66-a775-54be5ea96723","Type":"ContainerDied","Data":"8d98d857f7904b0b935249cb177d6416a9a0c36465b5f2850b59e86cbd2ae02d"}
Oct 30 19:15:02.245331 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.245206    4925 scope.go:122] "RemoveContainer" containerID="8d98d857f7904b0b935249cb177d6416a9a0c36465b5f2850b59e86cbd2ae02d"
Oct 30 19:15:02.246912 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.246887    4925 generic.go:356] "Generic (PLEG): container finished" podID="673b9cea-e53a-46be-968f-3ad97d7deb2d" containerID="eca3c87c7483e01b10dbaf2834bb5a645f1720ae1137299e31129461a7b20e51" exitCode=60
Oct 30 19:15:02.247057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.246984    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd" event={"ID":"673b9cea-e53a-46be-968f-3ad97d7deb2d","Type":"ContainerDied","Data":"eca3c87c7483e01b10dbaf2834bb5a645f1720ae1137299e31129461a7b20e51"}
Oct 30 19:15:02.247309 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.247289    4925 scope.go:122] "RemoveContainer" containerID="eca3c87c7483e01b10dbaf2834bb5a645f1720ae1137299e31129461a7b20e51"
Oct 30 19:15:02.249008 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.248986    4925 generic.go:356] "Generic (PLEG): container finished" podID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" containerID="5951bf8c202bb05a31343cce5c41ea620a136deab56ff005be31495ea243624f" exitCode=60
Oct 30 19:15:02.249177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.249033    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf" event={"ID":"71f8aae7-be84-42e4-b3f3-66d95a9e5579","Type":"ContainerDied","Data":"5951bf8c202bb05a31343cce5c41ea620a136deab56ff005be31495ea243624f"}
Oct 30 19:15:02.249324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.249308    4925 scope.go:122] "RemoveContainer" containerID="5951bf8c202bb05a31343cce5c41ea620a136deab56ff005be31495ea243624f"
Oct 30 19:15:02.490624 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.490592    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:02.873924 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.873898    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"]
Oct 30 19:15:02.875622 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:02.875261    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph" podUID="fe7bcca5-db06-4385-8f07-a402030e64b1" containerName="maas-api" containerID="cri-o://fee78f70ec9dc1f86d4c4bf1087464fc527b64a8278f4ff7f0629be813bd8f8e" gracePeriod=30
Oct 30 19:15:03.007445 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.007409    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:03.344085 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.343496    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:03.573395 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.573338    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:03.599254 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.599124    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd" event={"ID":"673b9cea-e53a-46be-968f-3ad97d7deb2d","Type":"ContainerStarted","Data":"60936601a30b4f3e6df7b2b556db95ee2f115c85526f4b6253fa4d2831330ee2"}
Oct 30 19:15:03.607461 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.607416    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf" event={"ID":"71f8aae7-be84-42e4-b3f3-66d95a9e5579","Type":"ContainerStarted","Data":"f21a4899c3c617352f29da9ae3866bf6ce656f66e7b5d635cf6c645293093a30"}
Oct 30 19:15:03.611900 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.611018    4925 generic.go:356] "Generic (PLEG): container finished" podID="fe7bcca5-db06-4385-8f07-a402030e64b1" containerID="fee78f70ec9dc1f86d4c4bf1087464fc527b64a8278f4ff7f0629be813bd8f8e" exitCode=0
Oct 30 19:15:03.611900 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.611261    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph" event={"ID":"fe7bcca5-db06-4385-8f07-a402030e64b1","Type":"ContainerDied","Data":"fee78f70ec9dc1f86d4c4bf1087464fc527b64a8278f4ff7f0629be813bd8f8e"}
Oct 30 19:15:03.614119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.614067    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t" event={"ID":"50839431-baf0-4a66-a775-54be5ea96723","Type":"ContainerStarted","Data":"e1edad17b25abda2eb53d02d55b31f45a0cca2ab464fc63546a0831c157255dd"}
Oct 30 19:15:03.614822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.614674    4925 scope.go:122] "RemoveContainer" containerID="e1edad17b25abda2eb53d02d55b31f45a0cca2ab464fc63546a0831c157255dd"
Oct 30 19:15:03.615349 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:03.615326    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t_odh-ai-gateway-infra(50839431-baf0-4a66-a775-54be5ea96723)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t" podUID="50839431-baf0-4a66-a775-54be5ea96723"
Oct 30 19:15:03.635217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.635167    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd" podStartSLOduration=3.635152757 podStartE2EDuration="3.635152757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:03.627476238 +0000 UTC m=+2325.343821326" watchObservedRunningTime="2026-10-30 19:15:03.635152757 +0000 UTC m=+2325.351497846"
Oct 30 19:15:03.745603 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.743974    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd" podUID="547ccaea-7848-46dd-a638-3411f31835f7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:03.758335 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.758235    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:15:03.798494 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:03.798449    4925 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50839431_baf0_4a66_a775_54be5ea96723.slice/crio-conmon-e1edad17b25abda2eb53d02d55b31f45a0cca2ab464fc63546a0831c157255dd.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50839431_baf0_4a66_a775_54be5ea96723.slice/crio-e1edad17b25abda2eb53d02d55b31f45a0cca2ab464fc63546a0831c157255dd.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673b9cea_e53a_46be_968f_3ad97d7deb2d.slice/crio-60936601a30b4f3e6df7b2b556db95ee2f115c85526f4b6253fa4d2831330ee2.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673b9cea_e53a_46be_968f_3ad97d7deb2d.slice/crio-conmon-60936601a30b4f3e6df7b2b556db95ee2f115c85526f4b6253fa4d2831330ee2.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 19:15:03.800211 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.800167    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf" podStartSLOduration=3.80015459 podStartE2EDuration="3.80015459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:03.728950695 +0000 UTC m=+2325.445295801" watchObservedRunningTime="2026-10-30 19:15:03.80015459 +0000 UTC m=+2325.516499666"
Oct 30 19:15:03.873147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.873057    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:03.890363 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.890334    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe7bcca5-db06-4385-8f07-a402030e64b1-kube-api-access-vvt72\" (UniqueName: \"kubernetes.io/projected/fe7bcca5-db06-4385-8f07-a402030e64b1-kube-api-access-vvt72\") pod \"fe7bcca5-db06-4385-8f07-a402030e64b1\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") "
Oct 30 19:15:03.890548 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.890488    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe7bcca5-db06-4385-8f07-a402030e64b1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe7bcca5-db06-4385-8f07-a402030e64b1-openshift-service-ca\") pod \"fe7bcca5-db06-4385-8f07-a402030e64b1\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") "
Oct 30 19:15:03.890638 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.890611    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-metrics-certs\") pod \"fe7bcca5-db06-4385-8f07-a402030e64b1\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") "
Oct 30 19:15:03.890917 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.890899    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls\") pod \"fe7bcca5-db06-4385-8f07-a402030e64b1\" (UID: \"fe7bcca5-db06-4385-8f07-a402030e64b1\") "
Oct 30 19:15:03.891065 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.890907    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe7bcca5-db06-4385-8f07-a402030e64b1-openshift-service-ca" pod "fe7bcca5-db06-4385-8f07-a402030e64b1" (UID: "fe7bcca5-db06-4385-8f07-a402030e64b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:03.891628 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.891580    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe7bcca5-db06-4385-8f07-a402030e64b1-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:03.893213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.893189    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-metrics-certs" pod "fe7bcca5-db06-4385-8f07-a402030e64b1" (UID: "fe7bcca5-db06-4385-8f07-a402030e64b1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:03.893580 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.893555    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe7bcca5-db06-4385-8f07-a402030e64b1-kube-api-access-vvt72" pod "fe7bcca5-db06-4385-8f07-a402030e64b1" (UID: "fe7bcca5-db06-4385-8f07-a402030e64b1"). InnerVolumeSpecName "kube-api-access-vvt72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:03.893777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.893756    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls" pod "fe7bcca5-db06-4385-8f07-a402030e64b1" (UID: "fe7bcca5-db06-4385-8f07-a402030e64b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:03.993480 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.993445    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:03.993646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.993482    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvt72\" (UniqueName: \"kubernetes.io/projected/fe7bcca5-db06-4385-8f07-a402030e64b1-kube-api-access-vvt72\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:03.993646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:03.993503    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe7bcca5-db06-4385-8f07-a402030e64b1-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:04.063322 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.063289    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:04.624354 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.624321    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"
Oct 30 19:15:04.624864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.624319    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph" event={"ID":"fe7bcca5-db06-4385-8f07-a402030e64b1","Type":"ContainerDied","Data":"e0aa0837a0334f0e30cd297241f1a793772e48f74966a50615e7af8faafab49e"}
Oct 30 19:15:04.624864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.624504    4925 scope.go:122] "RemoveContainer" containerID="fee78f70ec9dc1f86d4c4bf1087464fc527b64a8278f4ff7f0629be813bd8f8e"
Oct 30 19:15:04.626862 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.626835    4925 generic.go:356] "Generic (PLEG): container finished" podID="50839431-baf0-4a66-a775-54be5ea96723" containerID="e1edad17b25abda2eb53d02d55b31f45a0cca2ab464fc63546a0831c157255dd" exitCode=60
Oct 30 19:15:04.627049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.627029    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t" event={"ID":"50839431-baf0-4a66-a775-54be5ea96723","Type":"ContainerDied","Data":"e1edad17b25abda2eb53d02d55b31f45a0cca2ab464fc63546a0831c157255dd"}
Oct 30 19:15:04.627719 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.627699    4925 scope.go:122] "RemoveContainer" containerID="e1edad17b25abda2eb53d02d55b31f45a0cca2ab464fc63546a0831c157255dd"
Oct 30 19:15:04.628175 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:04.628070    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t_odh-ai-gateway-infra(50839431-baf0-4a66-a775-54be5ea96723)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t" podUID="50839431-baf0-4a66-a775-54be5ea96723"
Oct 30 19:15:04.630952 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.630924    4925 generic.go:356] "Generic (PLEG): container finished" podID="673b9cea-e53a-46be-968f-3ad97d7deb2d" containerID="60936601a30b4f3e6df7b2b556db95ee2f115c85526f4b6253fa4d2831330ee2" exitCode=60
Oct 30 19:15:04.631073 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.631019    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd" event={"ID":"673b9cea-e53a-46be-968f-3ad97d7deb2d","Type":"ContainerDied","Data":"60936601a30b4f3e6df7b2b556db95ee2f115c85526f4b6253fa4d2831330ee2"}
Oct 30 19:15:04.631631 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.631613    4925 scope.go:122] "RemoveContainer" containerID="60936601a30b4f3e6df7b2b556db95ee2f115c85526f4b6253fa4d2831330ee2"
Oct 30 19:15:04.631909 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:04.631888    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd_odh-ai-gateway-infra(673b9cea-e53a-46be-968f-3ad97d7deb2d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d"
Oct 30 19:15:04.635566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.635544    4925 generic.go:356] "Generic (PLEG): container finished" podID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" containerID="f21a4899c3c617352f29da9ae3866bf6ce656f66e7b5d635cf6c645293093a30" exitCode=60
Oct 30 19:15:04.635691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.635645    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf" event={"ID":"71f8aae7-be84-42e4-b3f3-66d95a9e5579","Type":"ContainerDied","Data":"f21a4899c3c617352f29da9ae3866bf6ce656f66e7b5d635cf6c645293093a30"}
Oct 30 19:15:04.635867 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.635851    4925 scope.go:122] "RemoveContainer" containerID="f21a4899c3c617352f29da9ae3866bf6ce656f66e7b5d635cf6c645293093a30"
Oct 30 19:15:04.636172 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:04.636148    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf_odh-ai-gateway-infra(71f8aae7-be84-42e4-b3f3-66d95a9e5579)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf" podUID="71f8aae7-be84-42e4-b3f3-66d95a9e5579"
Oct 30 19:15:04.638823 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.638803    4925 scope.go:122] "RemoveContainer" containerID="8d98d857f7904b0b935249cb177d6416a9a0c36465b5f2850b59e86cbd2ae02d"
Oct 30 19:15:04.657157 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.657139    4925 scope.go:122] "RemoveContainer" containerID="eca3c87c7483e01b10dbaf2834bb5a645f1720ae1137299e31129461a7b20e51"
Oct 30 19:15:04.674066 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.674022    4925 scope.go:122] "RemoveContainer" containerID="5951bf8c202bb05a31343cce5c41ea620a136deab56ff005be31495ea243624f"
Oct 30 19:15:04.678317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.678280    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:04.770151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.753336    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"]
Oct 30 19:15:04.770475 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.770424    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-e67d95-9b9575654-dbjph"]
Oct 30 19:15:04.869917 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:04.869886    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:05.087835 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:05.087311    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:05.648340 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:05.648307    4925 scope.go:122] "RemoveContainer" containerID="e1edad17b25abda2eb53d02d55b31f45a0cca2ab464fc63546a0831c157255dd"
Oct 30 19:15:05.649361 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:05.649328    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t_odh-ai-gateway-infra(50839431-baf0-4a66-a775-54be5ea96723)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t" podUID="50839431-baf0-4a66-a775-54be5ea96723"
Oct 30 19:15:05.650279 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:05.650256    4925 scope.go:122] "RemoveContainer" containerID="60936601a30b4f3e6df7b2b556db95ee2f115c85526f4b6253fa4d2831330ee2"
Oct 30 19:15:05.650650 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:05.650618    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd_odh-ai-gateway-infra(673b9cea-e53a-46be-968f-3ad97d7deb2d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d"
Oct 30 19:15:05.652267 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:05.652241    4925 scope.go:122] "RemoveContainer" containerID="f21a4899c3c617352f29da9ae3866bf6ce656f66e7b5d635cf6c645293093a30"
Oct 30 19:15:05.652664 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:05.652631    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf_odh-ai-gateway-infra(71f8aae7-be84-42e4-b3f3-66d95a9e5579)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf" podUID="71f8aae7-be84-42e4-b3f3-66d95a9e5579"
Oct 30 19:15:06.271171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:06.271068    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:06.425981 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:06.425930    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe7bcca5-db06-4385-8f07-a402030e64b1" path="/var/lib/kubelet/pods/fe7bcca5-db06-4385-8f07-a402030e64b1/volumes"
Oct 30 19:15:07.113135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:07.113064    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:07.556939 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:07.556899    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:07.761000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:07.760935    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:07.808946 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:07.808864    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"]
Oct 30 19:15:07.809567 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:07.809532    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc" podUID="778ba6a8-bfbe-4e63-ae97-bff72ebfb931" containerName="maas-api" containerID="cri-o://ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7" gracePeriod=30
Oct 30 19:15:07.829643 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:07.827835    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"]
Oct 30 19:15:07.829643 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:07.828152    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" podUID="b330c3ad-b62a-4146-9626-d8e9045512ff" containerName="istio-proxy" containerID="cri-o://d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d" gracePeriod=30
Oct 30 19:15:07.841977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:07.841949    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"]
Oct 30 19:15:07.842433 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:07.842404    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7" podUID="33b1140b-4cbd-49b0-921f-bbcc2986ebb5" containerName="istio-proxy" containerID="cri-o://90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47" gracePeriod=30
Oct 30 19:15:07.959888 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:07.959783    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:08.306047 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.305841    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:08.522559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.522521    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:08.525029 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.525004    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"]
Oct 30 19:15:08.526257 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.526233    4925 cpu_manager.go:414] "containerMap: removing container" podUID="fe7bcca5-db06-4385-8f07-a402030e64b1" containerName="maas-api"
Oct 30 19:15:08.526407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.526391    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe7bcca5-db06-4385-8f07-a402030e64b1" containerName="maas-api"
Oct 30 19:15:08.526838 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.526818    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe7bcca5-db06-4385-8f07-a402030e64b1" containerName="maas-api"
Oct 30 19:15:08.531843 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.531825    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"]
Oct 30 19:15:08.532070 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.532021    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:08.536163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.536141    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:15:08.573783 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.573761    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:15:08.668014 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.667985    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-openshift-service-ca\") pod \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") "
Oct 30 19:15:08.668214 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.668145    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-metrics-certs\" (UniqueName: \"kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-metrics-certs\") pod \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") "
Oct 30 19:15:08.668214 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.668199    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-maas-api-tls\") pod \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") "
Oct 30 19:15:08.668348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.668276    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-kube-api-access-2xmq8\" (UniqueName: \"kubernetes.io/projected/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-kube-api-access-2xmq8\") pod \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\" (UID: \"778ba6a8-bfbe-4e63-ae97-bff72ebfb931\") "
Oct 30 19:15:08.668438 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.668408    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-openshift-service-ca" pod "778ba6a8-bfbe-4e63-ae97-bff72ebfb931" (UID: "778ba6a8-bfbe-4e63-ae97-bff72ebfb931"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:08.668749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.668710    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w\" (UID: \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:08.669310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.669283    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w\" (UID: \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:08.669536 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.669507    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv6z9\" (UniqueName: \"kubernetes.io/projected/14fb2633-bb43-4ae3-8ab4-b21e2011df09-kube-api-access-pv6z9\") pod \"maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w\" (UID: \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:08.669768 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.669748    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:08.670697 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.670677    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-metrics-certs" pod "778ba6a8-bfbe-4e63-ae97-bff72ebfb931" (UID: "778ba6a8-bfbe-4e63-ae97-bff72ebfb931"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:08.671156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.671134    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-maas-api-tls" pod "778ba6a8-bfbe-4e63-ae97-bff72ebfb931" (UID: "778ba6a8-bfbe-4e63-ae97-bff72ebfb931"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:08.671618 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.671590    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-kube-api-access-2xmq8" pod "778ba6a8-bfbe-4e63-ae97-bff72ebfb931" (UID: "778ba6a8-bfbe-4e63-ae97-bff72ebfb931"). InnerVolumeSpecName "kube-api-access-2xmq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:08.685913 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.685888    4925 generic.go:356] "Generic (PLEG): container finished" podID="778ba6a8-bfbe-4e63-ae97-bff72ebfb931" containerID="ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7" exitCode=0
Oct 30 19:15:08.686031 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.685940    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"
Oct 30 19:15:08.686031 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.685991    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc" event={"ID":"778ba6a8-bfbe-4e63-ae97-bff72ebfb931","Type":"ContainerDied","Data":"ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7"}
Oct 30 19:15:08.686175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.686032    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc" event={"ID":"778ba6a8-bfbe-4e63-ae97-bff72ebfb931","Type":"ContainerDied","Data":"5ea34228c15facc7152293f211ced0f2e2d2fa137bafe1df2dd423ac06823fd7"}
Oct 30 19:15:08.686175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.686063    4925 scope.go:122] "RemoveContainer" containerID="ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7"
Oct 30 19:15:08.720810 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.720518    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:08.732474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.732452    4925 scope.go:122] "RemoveContainer" containerID="ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7"
Oct 30 19:15:08.732838 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:08.732816    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7\": container with ID starting with ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7 not found: ID does not exist" containerID="ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7"
Oct 30 19:15:08.732896 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.732847    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7"} err="failed to get container status \"ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7\": rpc error: code = NotFound desc = could not find container \"ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7\": container with ID starting with ee317d0a63c90138a2b550b1678d4aab926b848d770fdc1d08f94a7269637bc7 not found: ID does not exist"
Oct 30 19:15:08.740688 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.740663    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"]
Oct 30 19:15:08.745755 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.745726    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-baa504-98dff6594-knwbc"]
Oct 30 19:15:08.771593 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.771562    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w\" (UID: \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:08.771740 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.771658    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv6z9\" (UniqueName: \"kubernetes.io/projected/14fb2633-bb43-4ae3-8ab4-b21e2011df09-kube-api-access-pv6z9\") pod \"maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w\" (UID: \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:08.771929 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.771910    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w\" (UID: \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:08.772190 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.772170    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:08.772354 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.772340    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:08.772473 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.772452    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xmq8\" (UniqueName: \"kubernetes.io/projected/778ba6a8-bfbe-4e63-ae97-bff72ebfb931-kube-api-access-2xmq8\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:08.772473 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.772372    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w\" (UID: \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:08.775199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.775176    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w\" (UID: \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:08.780415 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.780387    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv6z9\" (UniqueName: \"kubernetes.io/projected/14fb2633-bb43-4ae3-8ab4-b21e2011df09-kube-api-access-pv6z9\") pod \"maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w\" (UID: \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:08.885950 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:08.885887    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:09.205471 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:09.205440    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"]
Oct 30 19:15:09.206187 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:15:09.206158    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14fb2633_bb43_4ae3_8ab4_b21e2011df09.slice/crio-25951c4d02c44615fc1fe47bf5a97015d9f8bd0e32a4ee3938cadbf5f43b0f8a WatchSource:0}: Error finding container 25951c4d02c44615fc1fe47bf5a97015d9f8bd0e32a4ee3938cadbf5f43b0f8a: Status 404 returned error can't find the container with id 25951c4d02c44615fc1fe47bf5a97015d9f8bd0e32a4ee3938cadbf5f43b0f8a
Oct 30 19:15:09.479800 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:09.479701    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:09.702238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:09.702201    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w" event={"ID":"14fb2633-bb43-4ae3-8ab4-b21e2011df09","Type":"ContainerStarted","Data":"685ce236d5e6f39746f7873192dd973e29a560eb840ac68345c14f8e6b7719af"}
Oct 30 19:15:09.702419 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:09.702266    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w" event={"ID":"14fb2633-bb43-4ae3-8ab4-b21e2011df09","Type":"ContainerStarted","Data":"25951c4d02c44615fc1fe47bf5a97015d9f8bd0e32a4ee3938cadbf5f43b0f8a"}
Oct 30 19:15:09.719375 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:09.719325    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w" podStartSLOduration=1.719311467 podStartE2EDuration="1.719311467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:09.718395747 +0000 UTC m=+2331.434740823" watchObservedRunningTime="2026-10-30 19:15:09.719311467 +0000 UTC m=+2331.435656542"
Oct 30 19:15:10.239555 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:10.239518    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:10.425450 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:10.425420    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="778ba6a8-bfbe-4e63-ae97-bff72ebfb931" path="/var/lib/kubelet/pods/778ba6a8-bfbe-4e63-ae97-bff72ebfb931/volumes"
Oct 30 19:15:10.713332 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:10.713296    4925 generic.go:356] "Generic (PLEG): container finished" podID="14fb2633-bb43-4ae3-8ab4-b21e2011df09" containerID="685ce236d5e6f39746f7873192dd973e29a560eb840ac68345c14f8e6b7719af" exitCode=0
Oct 30 19:15:10.713807 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:10.713380    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w" event={"ID":"14fb2633-bb43-4ae3-8ab4-b21e2011df09","Type":"ContainerDied","Data":"685ce236d5e6f39746f7873192dd973e29a560eb840ac68345c14f8e6b7719af"}
Oct 30 19:15:10.906123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:10.891709    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:11.085202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:11.083429    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:11.105327 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:11.103478    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"]
Oct 30 19:15:11.105327 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:11.103713    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb" podUID="f531c729-0f21-45d5-b635-e8613eda326b" containerName="istio-proxy" containerID="cri-o://aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44" gracePeriod=30
Oct 30 19:15:11.295121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:11.295023    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:11.497166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:11.490841    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:11.913040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:11.913014    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:11.968443 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:11.968402    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14fb2633-bb43-4ae3-8ab4-b21e2011df09-kube-api-access-pv6z9\" (UniqueName: \"kubernetes.io/projected/14fb2633-bb43-4ae3-8ab4-b21e2011df09-kube-api-access-pv6z9\") pod \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\" (UID: \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\") "
Oct 30 19:15:11.968651 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:11.968564    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-xks-ca\") pod \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\" (UID: \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\") "
Oct 30 19:15:11.968728 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:11.968650    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-openshift-ca\") pod \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\" (UID: \"14fb2633-bb43-4ae3-8ab4-b21e2011df09\") "
Oct 30 19:15:11.969750 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:11.969710    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-openshift-ca" pod "14fb2633-bb43-4ae3-8ab4-b21e2011df09" (UID: "14fb2633-bb43-4ae3-8ab4-b21e2011df09"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:11.972646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:11.972597    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14fb2633-bb43-4ae3-8ab4-b21e2011df09-kube-api-access-pv6z9" pod "14fb2633-bb43-4ae3-8ab4-b21e2011df09" (UID: "14fb2633-bb43-4ae3-8ab4-b21e2011df09"). InnerVolumeSpecName "kube-api-access-pv6z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:11.973090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:11.973058    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-xks-ca" pod "14fb2633-bb43-4ae3-8ab4-b21e2011df09" (UID: "14fb2633-bb43-4ae3-8ab4-b21e2011df09"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:12.017356 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:12.017325    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2" podUID="822c68de-9860-4a2d-b7ae-145bd319f117" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:12.017575 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:12.017544    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf" podUID="1aeb450f-292b-4274-8bf4-2e8b6e686d50" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:12.076399 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:12.073592    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pv6z9\" (UniqueName: \"kubernetes.io/projected/14fb2633-bb43-4ae3-8ab4-b21e2011df09-kube-api-access-pv6z9\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:12.076399 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:12.073633    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-xks-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:12.076399 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:12.073648    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/14fb2633-bb43-4ae3-8ab4-b21e2011df09-maas-api-openshift-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:12.096155 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:12.096063    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:12.829169 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:12.828239    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:12.834157 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:12.834079    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w" event={"ID":"14fb2633-bb43-4ae3-8ab4-b21e2011df09","Type":"ContainerDied","Data":"25951c4d02c44615fc1fe47bf5a97015d9f8bd0e32a4ee3938cadbf5f43b0f8a"}
Oct 30 19:15:12.834425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:12.834183    4925 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25951c4d02c44615fc1fe47bf5a97015d9f8bd0e32a4ee3938cadbf5f43b0f8a"
Oct 30 19:15:12.834543 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:12.834524    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"
Oct 30 19:15:13.202511 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.202182    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:15:13.208440 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.208417    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240189    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-socket\") pod \"b330c3ad-b62a-4146-9626-d8e9045512ff\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240253    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-podinfo\") pod \"b330c3ad-b62a-4146-9626-d8e9045512ff\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240299    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-token\" (UniqueName: \"kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-token\") pod \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240365    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-kube-api-access-k2npd\" (UniqueName: \"kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-kube-api-access-k2npd\") pod \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240412    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-token\" (UniqueName: \"kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-token\") pod \"b330c3ad-b62a-4146-9626-d8e9045512ff\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240463    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-credential-socket\") pod \"b330c3ad-b62a-4146-9626-d8e9045512ff\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240514    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-data\") pod \"b330c3ad-b62a-4146-9626-d8e9045512ff\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240546    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-certs\") pod \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240600    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-envoy\") pod \"b330c3ad-b62a-4146-9626-d8e9045512ff\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240658    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-credential-socket\") pod \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240756    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istiod-ca-cert\") pod \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240825    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b330c3ad-b62a-4146-9626-d8e9045512ff-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b330c3ad-b62a-4146-9626-d8e9045512ff-istiod-ca-cert\") pod \"b330c3ad-b62a-4146-9626-d8e9045512ff\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240876    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-envoy\") pod \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.240916    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-certs\") pod \"b330c3ad-b62a-4146-9626-d8e9045512ff\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.241033    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-podinfo\") pod \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.241137    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-kube-api-access-mtm69\" (UniqueName: \"kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-kube-api-access-mtm69\") pod \"b330c3ad-b62a-4146-9626-d8e9045512ff\" (UID: \"b330c3ad-b62a-4146-9626-d8e9045512ff\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.242062    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-data" pod "b330c3ad-b62a-4146-9626-d8e9045512ff" (UID: "b330c3ad-b62a-4146-9626-d8e9045512ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.242317    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-credential-socket" pod "b330c3ad-b62a-4146-9626-d8e9045512ff" (UID: "b330c3ad-b62a-4146-9626-d8e9045512ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.242371    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-credential-socket" pod "33b1140b-4cbd-49b0-921f-bbcc2986ebb5" (UID: "33b1140b-4cbd-49b0-921f-bbcc2986ebb5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.242438    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-certs" pod "33b1140b-4cbd-49b0-921f-bbcc2986ebb5" (UID: "33b1140b-4cbd-49b0-921f-bbcc2986ebb5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.242672    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-socket" pod "b330c3ad-b62a-4146-9626-d8e9045512ff" (UID: "b330c3ad-b62a-4146-9626-d8e9045512ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.242967    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-socket\") pod \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.243019    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istiod-ca-cert" pod "33b1140b-4cbd-49b0-921f-bbcc2986ebb5" (UID: "33b1140b-4cbd-49b0-921f-bbcc2986ebb5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.243055    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-data\") pod \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\" (UID: \"33b1140b-4cbd-49b0-921f-bbcc2986ebb5\") "
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.243270    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-data" pod "33b1140b-4cbd-49b0-921f-bbcc2986ebb5" (UID: "33b1140b-4cbd-49b0-921f-bbcc2986ebb5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.243452    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-socket" pod "33b1140b-4cbd-49b0-921f-bbcc2986ebb5" (UID: "33b1140b-4cbd-49b0-921f-bbcc2986ebb5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.243663    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-certs" pod "b330c3ad-b62a-4146-9626-d8e9045512ff" (UID: "b330c3ad-b62a-4146-9626-d8e9045512ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.243675    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b330c3ad-b62a-4146-9626-d8e9045512ff-istiod-ca-cert" pod "b330c3ad-b62a-4146-9626-d8e9045512ff" (UID: "b330c3ad-b62a-4146-9626-d8e9045512ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.244548    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.244571    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.244587    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.244602    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.244617    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.244639    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b330c3ad-b62a-4146-9626-d8e9045512ff-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.244654    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.244668    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.244682    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.251474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.244695    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.272348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.265221    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-envoy" pod "33b1140b-4cbd-49b0-921f-bbcc2986ebb5" (UID: "33b1140b-4cbd-49b0-921f-bbcc2986ebb5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:13.272348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.265372    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-token" pod "b330c3ad-b62a-4146-9626-d8e9045512ff" (UID: "b330c3ad-b62a-4146-9626-d8e9045512ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:13.272348 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.266442    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-podinfo" pod "33b1140b-4cbd-49b0-921f-bbcc2986ebb5" (UID: "33b1140b-4cbd-49b0-921f-bbcc2986ebb5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:15:13.272711 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.272671    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-podinfo" pod "b330c3ad-b62a-4146-9626-d8e9045512ff" (UID: "b330c3ad-b62a-4146-9626-d8e9045512ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:15:13.273005 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.272910    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-kube-api-access-k2npd" pod "33b1140b-4cbd-49b0-921f-bbcc2986ebb5" (UID: "33b1140b-4cbd-49b0-921f-bbcc2986ebb5"). InnerVolumeSpecName "kube-api-access-k2npd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:13.273005 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.272957    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-kube-api-access-mtm69" pod "b330c3ad-b62a-4146-9626-d8e9045512ff" (UID: "b330c3ad-b62a-4146-9626-d8e9045512ff"). InnerVolumeSpecName "kube-api-access-mtm69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:13.273163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.273046    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-token" pod "33b1140b-4cbd-49b0-921f-bbcc2986ebb5" (UID: "33b1140b-4cbd-49b0-921f-bbcc2986ebb5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:13.275071 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.274873    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-envoy" pod "b330c3ad-b62a-4146-9626-d8e9045512ff" (UID: "b330c3ad-b62a-4146-9626-d8e9045512ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:13.327410 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:13.327367    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:15:13.346880 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.346840    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.347028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.346886    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtm69\" (UniqueName: \"kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-kube-api-access-mtm69\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.347028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.346905    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.347028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.346922    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.347028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.346937    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2npd\" (UniqueName: \"kubernetes.io/projected/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-kube-api-access-k2npd\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.347028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.346953    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.347028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.346968    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b330c3ad-b62a-4146-9626-d8e9045512ff-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.347028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.346985    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33b1140b-4cbd-49b0-921f-bbcc2986ebb5-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.674715 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.674656    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:13.809128 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.807685    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd" podUID="547ccaea-7848-46dd-a638-3411f31835f7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:13.824251 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.823773    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"
Oct 30 19:15:13.852016 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.851847    4925 generic.go:356] "Generic (PLEG): container finished" podID="1aeb450f-292b-4274-8bf4-2e8b6e686d50" containerID="bae428d9a99cff9cc2a085b7b2ab681152e6d4c61d95b279f77e6b1dfa6f80cb" exitCode=0
Oct 30 19:15:13.852016 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.851939    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf" event={"ID":"1aeb450f-292b-4274-8bf4-2e8b6e686d50","Type":"ContainerDied","Data":"bae428d9a99cff9cc2a085b7b2ab681152e6d4c61d95b279f77e6b1dfa6f80cb"}
Oct 30 19:15:13.856239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.855832    4925 generic.go:356] "Generic (PLEG): container finished" podID="33b1140b-4cbd-49b0-921f-bbcc2986ebb5" containerID="90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47" exitCode=0
Oct 30 19:15:13.856239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.855901    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"
Oct 30 19:15:13.856239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.856007    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7" event={"ID":"33b1140b-4cbd-49b0-921f-bbcc2986ebb5","Type":"ContainerDied","Data":"90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47"}
Oct 30 19:15:13.856239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.856064    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7" event={"ID":"33b1140b-4cbd-49b0-921f-bbcc2986ebb5","Type":"ContainerDied","Data":"dacb012feb6aefa5f11e41dbe9b0023037628a8850e2241f0710dc3be3b2b97a"}
Oct 30 19:15:13.856239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.856124    4925 scope.go:122] "RemoveContainer" containerID="90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47"
Oct 30 19:15:13.863149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.862143    4925 generic.go:356] "Generic (PLEG): container finished" podID="b330c3ad-b62a-4146-9626-d8e9045512ff" containerID="d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d" exitCode=0
Oct 30 19:15:13.863149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.862276    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" event={"ID":"b330c3ad-b62a-4146-9626-d8e9045512ff","Type":"ContainerDied","Data":"d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d"}
Oct 30 19:15:13.863149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.862327    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt" event={"ID":"b330c3ad-b62a-4146-9626-d8e9045512ff","Type":"ContainerDied","Data":"f621c05a1abcd8a5c14f66dad2cd25536574f0d4c5e3ae2afc9567650b7cd326"}
Oct 30 19:15:13.863149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.862422    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"
Oct 30 19:15:13.864710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.863406    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/822c68de-9860-4a2d-b7ae-145bd319f117-kube-api-access-vzhg7\" (UniqueName: \"kubernetes.io/projected/822c68de-9860-4a2d-b7ae-145bd319f117-kube-api-access-vzhg7\") pod \"822c68de-9860-4a2d-b7ae-145bd319f117\" (UID: \"822c68de-9860-4a2d-b7ae-145bd319f117\") "
Oct 30 19:15:13.864710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.863756    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/822c68de-9860-4a2d-b7ae-145bd319f117-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/822c68de-9860-4a2d-b7ae-145bd319f117-plugins-config-volume\") pod \"822c68de-9860-4a2d-b7ae-145bd319f117\" (UID: \"822c68de-9860-4a2d-b7ae-145bd319f117\") "
Oct 30 19:15:13.864886 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.864782    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/822c68de-9860-4a2d-b7ae-145bd319f117-plugins-config-volume" pod "822c68de-9860-4a2d-b7ae-145bd319f117" (UID: "822c68de-9860-4a2d-b7ae-145bd319f117"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:13.867836 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.867806    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/822c68de-9860-4a2d-b7ae-145bd319f117-kube-api-access-vzhg7" pod "822c68de-9860-4a2d-b7ae-145bd319f117" (UID: "822c68de-9860-4a2d-b7ae-145bd319f117"). InnerVolumeSpecName "kube-api-access-vzhg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:13.869962 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.869692    4925 generic.go:356] "Generic (PLEG): container finished" podID="822c68de-9860-4a2d-b7ae-145bd319f117" containerID="8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8" exitCode=0
Oct 30 19:15:13.869962 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.869824    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2" event={"ID":"822c68de-9860-4a2d-b7ae-145bd319f117","Type":"ContainerDied","Data":"8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8"}
Oct 30 19:15:13.869962 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.869865    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2" event={"ID":"822c68de-9860-4a2d-b7ae-145bd319f117","Type":"ContainerDied","Data":"2ac2dfb825fa76386b2074e4edc1ee492eaf880f4b87e7581fd470459131f57b"}
Oct 30 19:15:13.869962 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.869936    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"
Oct 30 19:15:13.901941 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.901910    4925 scope.go:122] "RemoveContainer" containerID="90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47"
Oct 30 19:15:13.902543 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:13.902514    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47\": container with ID starting with 90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47 not found: ID does not exist" containerID="90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47"
Oct 30 19:15:13.902666 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.902554    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47"} err="failed to get container status \"90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47\": rpc error: code = NotFound desc = could not find container \"90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47\": container with ID starting with 90a22934c25591399714ac0d4e48ff79644a84f1fd033569e293956939c72d47 not found: ID does not exist"
Oct 30 19:15:13.902666 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.902577    4925 scope.go:122] "RemoveContainer" containerID="d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d"
Oct 30 19:15:13.904836 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.904813    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"]
Oct 30 19:15:13.919747 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.919723    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-9bth7"]
Oct 30 19:15:13.927173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.927147    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"]
Oct 30 19:15:13.932497 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.932474    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-e51c10-openshift-default-54b5584fb5-txmmt"]
Oct 30 19:15:13.936110 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.936077    4925 scope.go:122] "RemoveContainer" containerID="d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d"
Oct 30 19:15:13.936490 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:13.936464    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d\": container with ID starting with d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d not found: ID does not exist" containerID="d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d"
Oct 30 19:15:13.936564 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.936500    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d"} err="failed to get container status \"d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d\": rpc error: code = NotFound desc = could not find container \"d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d\": container with ID starting with d5fec294fb3eb3cd8e10f3e9adfe726c9d59195e398e2ae7dfde5f161dbe195d not found: ID does not exist"
Oct 30 19:15:13.936564 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.936523    4925 scope.go:122] "RemoveContainer" containerID="8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8"
Oct 30 19:15:13.943559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.943537    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"]
Oct 30 19:15:13.948711 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.948683    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-e51c10-54cbdf79f4-wkpq2"]
Oct 30 19:15:13.955607 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.953621    4925 scope.go:122] "RemoveContainer" containerID="8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8"
Oct 30 19:15:13.955607 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:13.953977    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8\": container with ID starting with 8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8 not found: ID does not exist" containerID="8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8"
Oct 30 19:15:13.955607 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.954007    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8"} err="failed to get container status \"8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8\": rpc error: code = NotFound desc = could not find container \"8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8\": container with ID starting with 8372469ec113f3dcf3a5afb8525fbadd0087f5e2c04a4e3cfe0fef3acf6738a8 not found: ID does not exist"
Oct 30 19:15:13.966788 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.966722    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/822c68de-9860-4a2d-b7ae-145bd319f117-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:13.966788 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:13.966753    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzhg7\" (UniqueName: \"kubernetes.io/projected/822c68de-9860-4a2d-b7ae-145bd319f117-kube-api-access-vzhg7\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:14.087723 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.087652    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"
Oct 30 19:15:14.175072 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.175033    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1aeb450f-292b-4274-8bf4-2e8b6e686d50-kube-api-access-dvbrz\" (UniqueName: \"kubernetes.io/projected/1aeb450f-292b-4274-8bf4-2e8b6e686d50-kube-api-access-dvbrz\") pod \"1aeb450f-292b-4274-8bf4-2e8b6e686d50\" (UID: \"1aeb450f-292b-4274-8bf4-2e8b6e686d50\") "
Oct 30 19:15:14.175298 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.175235    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1aeb450f-292b-4274-8bf4-2e8b6e686d50-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1aeb450f-292b-4274-8bf4-2e8b6e686d50-plugins-config-volume\") pod \"1aeb450f-292b-4274-8bf4-2e8b6e686d50\" (UID: \"1aeb450f-292b-4274-8bf4-2e8b6e686d50\") "
Oct 30 19:15:14.175948 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.175919    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1aeb450f-292b-4274-8bf4-2e8b6e686d50-plugins-config-volume" pod "1aeb450f-292b-4274-8bf4-2e8b6e686d50" (UID: "1aeb450f-292b-4274-8bf4-2e8b6e686d50"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:14.178036 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.178011    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aeb450f-292b-4274-8bf4-2e8b6e686d50-kube-api-access-dvbrz" pod "1aeb450f-292b-4274-8bf4-2e8b6e686d50" (UID: "1aeb450f-292b-4274-8bf4-2e8b6e686d50"). InnerVolumeSpecName "kube-api-access-dvbrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:14.277052 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.277020    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvbrz\" (UniqueName: \"kubernetes.io/projected/1aeb450f-292b-4274-8bf4-2e8b6e686d50-kube-api-access-dvbrz\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:14.277052 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.277051    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1aeb450f-292b-4274-8bf4-2e8b6e686d50-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:14.396182 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.391695    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:14.418509 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.415912    4925 scope.go:122] "RemoveContainer" containerID="f21a4899c3c617352f29da9ae3866bf6ce656f66e7b5d635cf6c645293093a30"
Oct 30 19:15:14.444765 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.444689    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33b1140b-4cbd-49b0-921f-bbcc2986ebb5" path="/var/lib/kubelet/pods/33b1140b-4cbd-49b0-921f-bbcc2986ebb5/volumes"
Oct 30 19:15:14.445594 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.445570    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="822c68de-9860-4a2d-b7ae-145bd319f117" path="/var/lib/kubelet/pods/822c68de-9860-4a2d-b7ae-145bd319f117/volumes"
Oct 30 19:15:14.446310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.446289    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b330c3ad-b62a-4146-9626-d8e9045512ff" path="/var/lib/kubelet/pods/b330c3ad-b62a-4146-9626-d8e9045512ff/volumes"
Oct 30 19:15:14.882646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.882625    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"
Oct 30 19:15:14.882796 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.882673    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf" event={"ID":"1aeb450f-292b-4274-8bf4-2e8b6e686d50","Type":"ContainerDied","Data":"de5b87ffe295afb944bd5aa2b7bdef393f181324b84f445d747dde1ef98c8fd6"}
Oct 30 19:15:14.882796 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.882730    4925 scope.go:122] "RemoveContainer" containerID="bae428d9a99cff9cc2a085b7b2ab681152e6d4c61d95b279f77e6b1dfa6f80cb"
Oct 30 19:15:14.886755 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.886719    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf" event={"ID":"71f8aae7-be84-42e4-b3f3-66d95a9e5579","Type":"ContainerStarted","Data":"843fcce234d3703f9d381896afa2b8001745c07d7d472195530ca63d0743f2a5"}
Oct 30 19:15:14.908235 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.908190    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"]
Oct 30 19:15:14.912424 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:14.912401    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-e51c10-84d8cf95c5-ncgpf"]
Oct 30 19:15:15.270299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:15.265155    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:15.415236 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:15.415211    4925 scope.go:122] "RemoveContainer" containerID="60936601a30b4f3e6df7b2b556db95ee2f115c85526f4b6253fa4d2831330ee2"
Oct 30 19:15:15.415611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:15.415341    4925 scope.go:122] "RemoveContainer" containerID="e1edad17b25abda2eb53d02d55b31f45a0cca2ab464fc63546a0831c157255dd"
Oct 30 19:15:15.476684 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:15.476551    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:15.489493 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:15.489456    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"]
Oct 30 19:15:15.495057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:15.489929    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" podUID="e4469a92-c400-468a-8309-13f96c4509dc" containerName="istio-proxy" containerID="cri-o://bc4488f827c10ecf28a470a1d5f333b96968f1cad11abd24e88d28e6871a9ef2" gracePeriod=30
Oct 30 19:15:15.645125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:15.638821    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"]
Oct 30 19:15:15.704680 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:15.704570    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:15.996843 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:15.996016    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:16.033370 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.033232    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"]
Oct 30 19:15:16.039963 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.039931    4925 generic.go:356] "Generic (PLEG): container finished" podID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" containerID="843fcce234d3703f9d381896afa2b8001745c07d7d472195530ca63d0743f2a5" exitCode=60
Oct 30 19:15:16.040335 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.040309    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf" event={"ID":"71f8aae7-be84-42e4-b3f3-66d95a9e5579","Type":"ContainerDied","Data":"843fcce234d3703f9d381896afa2b8001745c07d7d472195530ca63d0743f2a5"}
Oct 30 19:15:16.040588 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.040571    4925 scope.go:122] "RemoveContainer" containerID="f21a4899c3c617352f29da9ae3866bf6ce656f66e7b5d635cf6c645293093a30"
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.128567    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"]
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.129877    4925 cpu_manager.go:414] "containerMap: removing container" podUID="778ba6a8-bfbe-4e63-ae97-bff72ebfb931" containerName="maas-api"
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.129899    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="778ba6a8-bfbe-4e63-ae97-bff72ebfb931" containerName="maas-api"
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.129918    4925 cpu_manager.go:414] "containerMap: removing container" podUID="822c68de-9860-4a2d-b7ae-145bd319f117" containerName="payload-processing"
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.129925    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="822c68de-9860-4a2d-b7ae-145bd319f117" containerName="payload-processing"
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.129948    4925 cpu_manager.go:414] "containerMap: removing container" podUID="b330c3ad-b62a-4146-9626-d8e9045512ff" containerName="istio-proxy"
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.129956    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b330c3ad-b62a-4146-9626-d8e9045512ff" containerName="istio-proxy"
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.130014    4925 cpu_manager.go:414] "containerMap: removing container" podUID="33b1140b-4cbd-49b0-921f-bbcc2986ebb5" containerName="istio-proxy"
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.130021    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33b1140b-4cbd-49b0-921f-bbcc2986ebb5" containerName="istio-proxy"
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.130032    4925 cpu_manager.go:414] "containerMap: removing container" podUID="14fb2633-bb43-4ae3-8ab4-b21e2011df09" containerName="revoke-keys"
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.130039    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14fb2633-bb43-4ae3-8ab4-b21e2011df09" containerName="revoke-keys"
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.130072    4925 cpu_manager.go:414] "containerMap: removing container" podUID="1aeb450f-292b-4274-8bf4-2e8b6e686d50" containerName="payload-pre-processing"
Oct 30 19:15:16.130090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.130081    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1aeb450f-292b-4274-8bf4-2e8b6e686d50" containerName="payload-pre-processing"
Oct 30 19:15:16.130834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.130395    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="778ba6a8-bfbe-4e63-ae97-bff72ebfb931" containerName="maas-api"
Oct 30 19:15:16.130834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.130412    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="822c68de-9860-4a2d-b7ae-145bd319f117" containerName="payload-processing"
Oct 30 19:15:16.130834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.130491    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="b330c3ad-b62a-4146-9626-d8e9045512ff" containerName="istio-proxy"
Oct 30 19:15:16.130834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.130519    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="33b1140b-4cbd-49b0-921f-bbcc2986ebb5" containerName="istio-proxy"
Oct 30 19:15:16.130834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.130528    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="14fb2633-bb43-4ae3-8ab4-b21e2011df09" containerName="revoke-keys"
Oct 30 19:15:16.130834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.130541    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="1aeb450f-292b-4274-8bf4-2e8b6e686d50" containerName="payload-pre-processing"
Oct 30 19:15:16.151834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.140472    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:16.151834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.147520    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"]
Oct 30 19:15:16.151834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.148638    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:15:16.442944 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.442633    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st\" (UID: \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:16.442944 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.442734    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd6s2\" (UniqueName: \"kubernetes.io/projected/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-kube-api-access-hd6s2\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st\" (UID: \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:16.442944 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.442858    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st\" (UID: \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:16.469616 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.463399    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1aeb450f-292b-4274-8bf4-2e8b6e686d50" path="/var/lib/kubelet/pods/1aeb450f-292b-4274-8bf4-2e8b6e686d50/volumes"
Oct 30 19:15:16.544781 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.544747    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st\" (UID: \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:16.544911 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.544869    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd6s2\" (UniqueName: \"kubernetes.io/projected/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-kube-api-access-hd6s2\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st\" (UID: \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:16.545043 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.545028    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st\" (UID: \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:16.545761 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.545735    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st\" (UID: \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:16.547790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.547763    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st\" (UID: \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:16.553445 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.553416    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd6s2\" (UniqueName: \"kubernetes.io/projected/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-kube-api-access-hd6s2\") pod \"maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st\" (UID: \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:16.592167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.591850    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"
Oct 30 19:15:16.729859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.703710    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:16.776347 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.759997    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71f8aae7-be84-42e4-b3f3-66d95a9e5579-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f8aae7-be84-42e4-b3f3-66d95a9e5579-maas-api-service-ca\") pod \"71f8aae7-be84-42e4-b3f3-66d95a9e5579\" (UID: \"71f8aae7-be84-42e4-b3f3-66d95a9e5579\") "
Oct 30 19:15:16.776347 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.760286    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71f8aae7-be84-42e4-b3f3-66d95a9e5579-kube-api-access-z277g\" (UniqueName: \"kubernetes.io/projected/71f8aae7-be84-42e4-b3f3-66d95a9e5579-kube-api-access-z277g\") pod \"71f8aae7-be84-42e4-b3f3-66d95a9e5579\" (UID: \"71f8aae7-be84-42e4-b3f3-66d95a9e5579\") "
Oct 30 19:15:16.776347 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.761449    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71f8aae7-be84-42e4-b3f3-66d95a9e5579-maas-api-service-ca" pod "71f8aae7-be84-42e4-b3f3-66d95a9e5579" (UID: "71f8aae7-be84-42e4-b3f3-66d95a9e5579"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:16.776347 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.765937    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71f8aae7-be84-42e4-b3f3-66d95a9e5579-kube-api-access-z277g" pod "71f8aae7-be84-42e4-b3f3-66d95a9e5579" (UID: "71f8aae7-be84-42e4-b3f3-66d95a9e5579"). InnerVolumeSpecName "kube-api-access-z277g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:16.830300 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.830270    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:16.862631 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.862602    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71f8aae7-be84-42e4-b3f3-66d95a9e5579-maas-api-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:16.862631 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.862631    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z277g\" (UniqueName: \"kubernetes.io/projected/71f8aae7-be84-42e4-b3f3-66d95a9e5579-kube-api-access-z277g\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:16.870482 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.870455    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:15:16.964090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.963540    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-envoy\") pod \"f531c729-0f21-45d5-b635-e8613eda326b\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") "
Oct 30 19:15:16.964630 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.964589    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-certs\") pod \"f531c729-0f21-45d5-b635-e8613eda326b\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") "
Oct 30 19:15:16.964751 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.964647    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-data\") pod \"f531c729-0f21-45d5-b635-e8613eda326b\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") "
Oct 30 19:15:16.964751 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.964678    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-kube-api-access-vqpk6\" (UniqueName: \"kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-kube-api-access-vqpk6\") pod \"f531c729-0f21-45d5-b635-e8613eda326b\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") "
Oct 30 19:15:16.964917 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.964901    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f531c729-0f21-45d5-b635-e8613eda326b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f531c729-0f21-45d5-b635-e8613eda326b-istio-podinfo\") pod \"f531c729-0f21-45d5-b635-e8613eda326b\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") "
Oct 30 19:15:16.964981 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.964944    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-credential-socket\") pod \"f531c729-0f21-45d5-b635-e8613eda326b\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") "
Oct 30 19:15:16.965035 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.965020    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-socket\") pod \"f531c729-0f21-45d5-b635-e8613eda326b\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") "
Oct 30 19:15:16.965225 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.965207    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-istio-token\" (UniqueName: \"kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-istio-token\") pod \"f531c729-0f21-45d5-b635-e8613eda326b\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") "
Oct 30 19:15:16.965412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.965393    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f531c729-0f21-45d5-b635-e8613eda326b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f531c729-0f21-45d5-b635-e8613eda326b-istiod-ca-cert\") pod \"f531c729-0f21-45d5-b635-e8613eda326b\" (UID: \"f531c729-0f21-45d5-b635-e8613eda326b\") "
Oct 30 19:15:16.966857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.965972    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-certs" pod "f531c729-0f21-45d5-b635-e8613eda326b" (UID: "f531c729-0f21-45d5-b635-e8613eda326b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:16.966857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.966142    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f531c729-0f21-45d5-b635-e8613eda326b-istiod-ca-cert" pod "f531c729-0f21-45d5-b635-e8613eda326b" (UID: "f531c729-0f21-45d5-b635-e8613eda326b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:16.966857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.966388    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-credential-socket" pod "f531c729-0f21-45d5-b635-e8613eda326b" (UID: "f531c729-0f21-45d5-b635-e8613eda326b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:16.966857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.966577    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-data" pod "f531c729-0f21-45d5-b635-e8613eda326b" (UID: "f531c729-0f21-45d5-b635-e8613eda326b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:16.966857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.966589    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:16.966857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.966610    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:16.966857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.966624    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f531c729-0f21-45d5-b635-e8613eda326b-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:16.966857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.966588    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-socket" pod "f531c729-0f21-45d5-b635-e8613eda326b" (UID: "f531c729-0f21-45d5-b635-e8613eda326b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:16.966857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.966783    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-envoy" pod "f531c729-0f21-45d5-b635-e8613eda326b" (UID: "f531c729-0f21-45d5-b635-e8613eda326b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:16.970243 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.969311    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-istio-token" pod "f531c729-0f21-45d5-b635-e8613eda326b" (UID: "f531c729-0f21-45d5-b635-e8613eda326b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:16.970243 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.969607    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-kube-api-access-vqpk6" pod "f531c729-0f21-45d5-b635-e8613eda326b" (UID: "f531c729-0f21-45d5-b635-e8613eda326b"). InnerVolumeSpecName "kube-api-access-vqpk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:16.971815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:16.971792    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f531c729-0f21-45d5-b635-e8613eda326b-istio-podinfo" pod "f531c729-0f21-45d5-b635-e8613eda326b" (UID: "f531c729-0f21-45d5-b635-e8613eda326b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:15:17.052200 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.052124    4925 generic.go:356] "Generic (PLEG): container finished" podID="50839431-baf0-4a66-a775-54be5ea96723" containerID="910d359c432f139bf722e588ab46abb3342c850d66845c75f747a63d27e321c5" exitCode=60
Oct 30 19:15:17.052345 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.052210    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t" event={"ID":"50839431-baf0-4a66-a775-54be5ea96723","Type":"ContainerDied","Data":"910d359c432f139bf722e588ab46abb3342c850d66845c75f747a63d27e321c5"}
Oct 30 19:15:17.052345 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.052272    4925 scope.go:122] "RemoveContainer" containerID="e1edad17b25abda2eb53d02d55b31f45a0cca2ab464fc63546a0831c157255dd"
Oct 30 19:15:17.052728 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.052703    4925 scope.go:122] "RemoveContainer" containerID="910d359c432f139bf722e588ab46abb3342c850d66845c75f747a63d27e321c5"
Oct 30 19:15:17.053184 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:17.053149    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t_odh-ai-gateway-infra(50839431-baf0-4a66-a775-54be5ea96723)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t" podUID="50839431-baf0-4a66-a775-54be5ea96723"
Oct 30 19:15:17.053876 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.053854    4925 generic.go:356] "Generic (PLEG): container finished" podID="f531c729-0f21-45d5-b635-e8613eda326b" containerID="aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44" exitCode=0
Oct 30 19:15:17.053979 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.053894    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"
Oct 30 19:15:17.053979 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.053936    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb" event={"ID":"f531c729-0f21-45d5-b635-e8613eda326b","Type":"ContainerDied","Data":"aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44"}
Oct 30 19:15:17.053979 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.053976    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb" event={"ID":"f531c729-0f21-45d5-b635-e8613eda326b","Type":"ContainerDied","Data":"ada1c36bb29c22cdbf750ca442e9ed90a82529001ef67aa4962bad21e4817a52"}
Oct 30 19:15:17.056040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.056014    4925 generic.go:356] "Generic (PLEG): container finished" podID="673b9cea-e53a-46be-968f-3ad97d7deb2d" containerID="3fa0a674aa529893f1d07fd5284d3e10487c7b1118cb899267625b24b0a89ab4" exitCode=60
Oct 30 19:15:17.056162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.056042    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd" event={"ID":"673b9cea-e53a-46be-968f-3ad97d7deb2d","Type":"ContainerDied","Data":"3fa0a674aa529893f1d07fd5284d3e10487c7b1118cb899267625b24b0a89ab4"}
Oct 30 19:15:17.057310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.056476    4925 scope.go:122] "RemoveContainer" containerID="3fa0a674aa529893f1d07fd5284d3e10487c7b1118cb899267625b24b0a89ab4"
Oct 30 19:15:17.061244 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:17.059579    4925 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd_odh-ai-gateway-infra(673b9cea-e53a-46be-968f-3ad97d7deb2d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d"
Oct 30 19:15:17.082088 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.082064    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.082255 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.082237    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqpk6\" (UniqueName: \"kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-kube-api-access-vqpk6\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.082369 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.082356    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f531c729-0f21-45d5-b635-e8613eda326b-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.082470 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.082455    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.082571 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.082556    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f531c729-0f21-45d5-b635-e8613eda326b-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.082655 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.082643    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f531c729-0f21-45d5-b635-e8613eda326b-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.085251 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.085220    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf" event={"ID":"71f8aae7-be84-42e4-b3f3-66d95a9e5579","Type":"ContainerDied","Data":"3f640ac526adeea8c1589de320c3b11e63cad2dd9f2e739813cf8052f28f3f3a"}
Oct 30 19:15:17.085364 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.085354    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"
Oct 30 19:15:17.090729 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.090710    4925 scope.go:122] "RemoveContainer" containerID="aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44"
Oct 30 19:15:17.111322 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.111298    4925 scope.go:122] "RemoveContainer" containerID="aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44"
Oct 30 19:15:17.111723 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:17.111689    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44\": container with ID starting with aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44 not found: ID does not exist" containerID="aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44"
Oct 30 19:15:17.111937 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.111906    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44"} err="failed to get container status \"aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44\": rpc error: code = NotFound desc = could not find container \"aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44\": container with ID starting with aa39d4feb83d1007c1f2fc9c8f0c95086a6637cf807bc718ff1fdd10302b2c44 not found: ID does not exist"
Oct 30 19:15:17.111937 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.111938    4925 scope.go:122] "RemoveContainer" containerID="60936601a30b4f3e6df7b2b556db95ee2f115c85526f4b6253fa4d2831330ee2"
Oct 30 19:15:17.126960 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.126939    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"]
Oct 30 19:15:17.128366 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.128344    4925 scope.go:122] "RemoveContainer" containerID="843fcce234d3703f9d381896afa2b8001745c07d7d472195530ca63d0743f2a5"
Oct 30 19:15:17.131348 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:15:17.131321    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d3dcae_cfe9_4e5e_b9c2_0e2b67f30964.slice/crio-15799b5fd52d5f0910e46467a480545b9774128cf1425af8a0d333879157d19f WatchSource:0}: Error finding container 15799b5fd52d5f0910e46467a480545b9774128cf1425af8a0d333879157d19f: Status 404 returned error can't find the container with id 15799b5fd52d5f0910e46467a480545b9774128cf1425af8a0d333879157d19f
Oct 30 19:15:17.131947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.131843    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"]
Oct 30 19:15:17.136323 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.136301    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-e67d95-openshift-default-7db5c6f57c-cv8pb"]
Oct 30 19:15:17.150205 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.150179    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"]
Oct 30 19:15:17.154270 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.154248    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-2889dd-29889795-tfsmf"]
Oct 30 19:15:17.265613 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.265585    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:15:17.265746 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.265654    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" podUID="e4469a92-c400-468a-8309-13f96c4509dc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:15:17.850865 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:17.850769    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:18.085056 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.084954    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"]
Oct 30 19:15:18.104882 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.104844    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st" event={"ID":"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964","Type":"ContainerStarted","Data":"a7b348eee55a8c1008d8e00134da4a9199f315d9b35bd232c2a6439102173c19"}
Oct 30 19:15:18.105022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.104909    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st" event={"ID":"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964","Type":"ContainerStarted","Data":"15799b5fd52d5f0910e46467a480545b9774128cf1425af8a0d333879157d19f"}
Oct 30 19:15:18.110848 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.110823    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"]
Oct 30 19:15:18.154690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.154630    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st" podStartSLOduration=2.154616697 podStartE2EDuration="2.154616697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:18.149527087 +0000 UTC m=+2339.865872163" watchObservedRunningTime="2026-10-30 19:15:18.154616697 +0000 UTC m=+2339.870961770"
Oct 30 19:15:18.326775 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.326748    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"
Oct 30 19:15:18.414268 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.414238    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/673b9cea-e53a-46be-968f-3ad97d7deb2d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/673b9cea-e53a-46be-968f-3ad97d7deb2d-maas-api-service-ca\") pod \"673b9cea-e53a-46be-968f-3ad97d7deb2d\" (UID: \"673b9cea-e53a-46be-968f-3ad97d7deb2d\") "
Oct 30 19:15:18.414413 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.414399    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/673b9cea-e53a-46be-968f-3ad97d7deb2d-kube-api-access-nrqfk\" (UniqueName: \"kubernetes.io/projected/673b9cea-e53a-46be-968f-3ad97d7deb2d-kube-api-access-nrqfk\") pod \"673b9cea-e53a-46be-968f-3ad97d7deb2d\" (UID: \"673b9cea-e53a-46be-968f-3ad97d7deb2d\") "
Oct 30 19:15:18.414610 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.414586    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/673b9cea-e53a-46be-968f-3ad97d7deb2d-maas-api-service-ca" pod "673b9cea-e53a-46be-968f-3ad97d7deb2d" (UID: "673b9cea-e53a-46be-968f-3ad97d7deb2d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:18.415346 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.415166    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/673b9cea-e53a-46be-968f-3ad97d7deb2d-maas-api-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:18.417143 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.417114    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/673b9cea-e53a-46be-968f-3ad97d7deb2d-kube-api-access-nrqfk" pod "673b9cea-e53a-46be-968f-3ad97d7deb2d" (UID: "673b9cea-e53a-46be-968f-3ad97d7deb2d"). InnerVolumeSpecName "kube-api-access-nrqfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:18.428912 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.428885    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" path="/var/lib/kubelet/pods/71f8aae7-be84-42e4-b3f3-66d95a9e5579/volumes"
Oct 30 19:15:18.429644 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.429624    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f531c729-0f21-45d5-b635-e8613eda326b" path="/var/lib/kubelet/pods/f531c729-0f21-45d5-b635-e8613eda326b/volumes"
Oct 30 19:15:18.435037 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.435014    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"
Oct 30 19:15:18.516281 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.516244    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50839431-baf0-4a66-a775-54be5ea96723-kube-api-access-g4tg5\" (UniqueName: \"kubernetes.io/projected/50839431-baf0-4a66-a775-54be5ea96723-kube-api-access-g4tg5\") pod \"50839431-baf0-4a66-a775-54be5ea96723\" (UID: \"50839431-baf0-4a66-a775-54be5ea96723\") "
Oct 30 19:15:18.516512 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.516496    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50839431-baf0-4a66-a775-54be5ea96723-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50839431-baf0-4a66-a775-54be5ea96723-maas-api-service-ca\") pod \"50839431-baf0-4a66-a775-54be5ea96723\" (UID: \"50839431-baf0-4a66-a775-54be5ea96723\") "
Oct 30 19:15:18.516881 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.516858    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50839431-baf0-4a66-a775-54be5ea96723-maas-api-service-ca" pod "50839431-baf0-4a66-a775-54be5ea96723" (UID: "50839431-baf0-4a66-a775-54be5ea96723"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:18.517326 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.517301    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50839431-baf0-4a66-a775-54be5ea96723-maas-api-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:18.517403 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.517332    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrqfk\" (UniqueName: \"kubernetes.io/projected/673b9cea-e53a-46be-968f-3ad97d7deb2d-kube-api-access-nrqfk\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:18.518699 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.518673    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50839431-baf0-4a66-a775-54be5ea96723-kube-api-access-g4tg5" pod "50839431-baf0-4a66-a775-54be5ea96723" (UID: "50839431-baf0-4a66-a775-54be5ea96723"). InnerVolumeSpecName "kube-api-access-g4tg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:18.541465 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.541441    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"]
Oct 30 19:15:18.553153 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.553129    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"]
Oct 30 19:15:18.622129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.622009    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4tg5\" (UniqueName: \"kubernetes.io/projected/50839431-baf0-4a66-a775-54be5ea96723-kube-api-access-g4tg5\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:18.695326 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.695288    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:18.886718 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:18.886248    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:19.119366 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.119314    4925 generic.go:356] "Generic (PLEG): container finished" podID="11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964" containerID="a7b348eee55a8c1008d8e00134da4a9199f315d9b35bd232c2a6439102173c19" exitCode=0
Oct 30 19:15:19.119571 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.119448    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st" event={"ID":"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964","Type":"ContainerDied","Data":"a7b348eee55a8c1008d8e00134da4a9199f315d9b35bd232c2a6439102173c19"}
Oct 30 19:15:19.120979 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.120961    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"
Oct 30 19:15:19.121079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.120959    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd" event={"ID":"673b9cea-e53a-46be-968f-3ad97d7deb2d","Type":"ContainerDied","Data":"d44ea92c6ea3e9bcb845866c1da78f174e2f990e704b166bdeaaacaa7d48275c"}
Oct 30 19:15:19.121161 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.121136    4925 scope.go:122] "RemoveContainer" containerID="3fa0a674aa529893f1d07fd5284d3e10487c7b1118cb899267625b24b0a89ab4"
Oct 30 19:15:19.123087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.122808    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t" event={"ID":"50839431-baf0-4a66-a775-54be5ea96723","Type":"ContainerDied","Data":"c17af425cbba80882012d2f67022438585ab3733982734678cd6cd3c66e6fa5a"}
Oct 30 19:15:19.123087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.122880    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"
Oct 30 19:15:19.134684 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.134649    4925 scope.go:122] "RemoveContainer" containerID="910d359c432f139bf722e588ab46abb3342c850d66845c75f747a63d27e321c5"
Oct 30 19:15:19.159805 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.159781    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"]
Oct 30 19:15:19.170549 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.170527    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-4a477d-29889795-2rldd"]
Oct 30 19:15:19.187279 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.187250    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"]
Oct 30 19:15:19.191732 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.191710    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-ca2cd5-29889795-vfs6t"]
Oct 30 19:15:19.608117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:19.607220    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:20.298197 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.292784    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:20.424036 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.424012    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:20.425576 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.425555    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50839431-baf0-4a66-a775-54be5ea96723" path="/var/lib/kubelet/pods/50839431-baf0-4a66-a775-54be5ea96723/volumes"
Oct 30 19:15:20.426044 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.426031    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d" path="/var/lib/kubelet/pods/673b9cea-e53a-46be-968f-3ad97d7deb2d/volumes"
Oct 30 19:15:20.480021 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.479987    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-openshift-ca\") pod \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\" (UID: \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\") "
Oct 30 19:15:20.480215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.480160    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-xks-ca\") pod \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\" (UID: \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\") "
Oct 30 19:15:20.480286 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.480246    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-kube-api-access-hd6s2\" (UniqueName: \"kubernetes.io/projected/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-kube-api-access-hd6s2\") pod \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\" (UID: \"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964\") "
Oct 30 19:15:20.480529 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.480500    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-openshift-ca" pod "11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964" (UID: "11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:20.481307 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.481283    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-openshift-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:20.483786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.483714    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-xks-ca" pod "11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964" (UID: "11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:20.483786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.483752    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-kube-api-access-hd6s2" pod "11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964" (UID: "11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964"). InnerVolumeSpecName "kube-api-access-hd6s2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:20.582463 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.582440    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-maas-api-xks-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:20.582572 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.582465    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd6s2\" (UniqueName: \"kubernetes.io/projected/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964-kube-api-access-hd6s2\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:20.639629 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.639537    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t" podUID="2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab" containerName="payload-processing" containerID="cri-o://15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f" gracePeriod=25
Oct 30 19:15:20.868629 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.868595    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"
Oct 30 19:15:20.988641 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.988592    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/547ccaea-7848-46dd-a638-3411f31835f7-kube-api-access-t8cks\" (UniqueName: \"kubernetes.io/projected/547ccaea-7848-46dd-a638-3411f31835f7-kube-api-access-t8cks\") pod \"547ccaea-7848-46dd-a638-3411f31835f7\" (UID: \"547ccaea-7848-46dd-a638-3411f31835f7\") "
Oct 30 19:15:20.988899 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.988753    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/547ccaea-7848-46dd-a638-3411f31835f7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547ccaea-7848-46dd-a638-3411f31835f7-plugins-config-volume\") pod \"547ccaea-7848-46dd-a638-3411f31835f7\" (UID: \"547ccaea-7848-46dd-a638-3411f31835f7\") "
Oct 30 19:15:20.990776 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.990748    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/547ccaea-7848-46dd-a638-3411f31835f7-plugins-config-volume" pod "547ccaea-7848-46dd-a638-3411f31835f7" (UID: "547ccaea-7848-46dd-a638-3411f31835f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:20.993641 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:20.993616    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/547ccaea-7848-46dd-a638-3411f31835f7-kube-api-access-t8cks" pod "547ccaea-7848-46dd-a638-3411f31835f7" (UID: "547ccaea-7848-46dd-a638-3411f31835f7"). InnerVolumeSpecName "kube-api-access-t8cks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:21.092334 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.092305    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8cks\" (UniqueName: \"kubernetes.io/projected/547ccaea-7848-46dd-a638-3411f31835f7-kube-api-access-t8cks\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.092334 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.092333    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/547ccaea-7848-46dd-a638-3411f31835f7-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.238278 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.238237    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:21.249997 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.249965    4925 generic.go:356] "Generic (PLEG): container finished" podID="e4469a92-c400-468a-8309-13f96c4509dc" containerID="bc4488f827c10ecf28a470a1d5f333b96968f1cad11abd24e88d28e6871a9ef2" exitCode=0
Oct 30 19:15:21.250145 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.250058    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" event={"ID":"e4469a92-c400-468a-8309-13f96c4509dc","Type":"ContainerDied","Data":"bc4488f827c10ecf28a470a1d5f333b96968f1cad11abd24e88d28e6871a9ef2"}
Oct 30 19:15:21.253019 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.252994    4925 generic.go:356] "Generic (PLEG): container finished" podID="547ccaea-7848-46dd-a638-3411f31835f7" containerID="158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae" exitCode=0
Oct 30 19:15:21.253164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.253038    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"
Oct 30 19:15:21.253238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.253166    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd" event={"ID":"547ccaea-7848-46dd-a638-3411f31835f7","Type":"ContainerDied","Data":"158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae"}
Oct 30 19:15:21.253238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.253208    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd" event={"ID":"547ccaea-7848-46dd-a638-3411f31835f7","Type":"ContainerDied","Data":"d5e288d90c35f47a3445c2410ba9f011fcba918849a077c6f56b85f8074d9bda"}
Oct 30 19:15:21.253336 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.253237    4925 scope.go:122] "RemoveContainer" containerID="158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae"
Oct 30 19:15:21.256911 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.256888    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st" event={"ID":"11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964","Type":"ContainerDied","Data":"15799b5fd52d5f0910e46467a480545b9774128cf1425af8a0d333879157d19f"}
Oct 30 19:15:21.257031 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.256931    4925 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15799b5fd52d5f0910e46467a480545b9774128cf1425af8a0d333879157d19f"
Oct 30 19:15:21.257031 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.256988    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"
Oct 30 19:15:21.265402 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.265381    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:15:21.273002 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.272982    4925 scope.go:122] "RemoveContainer" containerID="158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae"
Oct 30 19:15:21.274388 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:21.273393    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae\": container with ID starting with 158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae not found: ID does not exist" containerID="158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae"
Oct 30 19:15:21.274388 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.273455    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae"} err="failed to get container status \"158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae\": rpc error: code = NotFound desc = could not find container \"158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae\": container with ID starting with 158758c596af8326700d8b95c5b34aa34808e5eced1bbb4dff889305fdd24bae not found: ID does not exist"
Oct 30 19:15:21.288266 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.288239    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"]
Oct 30 19:15:21.295730 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.295708    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-e67d95-944b9d9d-v24xd"]
Oct 30 19:15:21.406023 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.404695    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-certs\") pod \"e4469a92-c400-468a-8309-13f96c4509dc\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") "
Oct 30 19:15:21.406023 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.404756    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e4469a92-c400-468a-8309-13f96c4509dc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4469a92-c400-468a-8309-13f96c4509dc-istio-podinfo\") pod \"e4469a92-c400-468a-8309-13f96c4509dc\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") "
Oct 30 19:15:21.406023 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.404776    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-socket\") pod \"e4469a92-c400-468a-8309-13f96c4509dc\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") "
Oct 30 19:15:21.406023 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.404861    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-envoy\") pod \"e4469a92-c400-468a-8309-13f96c4509dc\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") "
Oct 30 19:15:21.406023 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.404885    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-kube-api-access-wgz2f\" (UniqueName: \"kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-kube-api-access-wgz2f\") pod \"e4469a92-c400-468a-8309-13f96c4509dc\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") "
Oct 30 19:15:21.406023 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.404925    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-istio-token\" (UniqueName: \"kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-istio-token\") pod \"e4469a92-c400-468a-8309-13f96c4509dc\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") "
Oct 30 19:15:21.406023 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.404953    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-data\") pod \"e4469a92-c400-468a-8309-13f96c4509dc\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") "
Oct 30 19:15:21.406023 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.405034    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4469a92-c400-468a-8309-13f96c4509dc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4469a92-c400-468a-8309-13f96c4509dc-istiod-ca-cert\") pod \"e4469a92-c400-468a-8309-13f96c4509dc\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") "
Oct 30 19:15:21.406023 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.405269    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-credential-socket\") pod \"e4469a92-c400-468a-8309-13f96c4509dc\" (UID: \"e4469a92-c400-468a-8309-13f96c4509dc\") "
Oct 30 19:15:21.406691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.406665    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-credential-socket" pod "e4469a92-c400-468a-8309-13f96c4509dc" (UID: "e4469a92-c400-468a-8309-13f96c4509dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:21.407743 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.407718    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-socket" pod "e4469a92-c400-468a-8309-13f96c4509dc" (UID: "e4469a92-c400-468a-8309-13f96c4509dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:21.408112 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.408073    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-data" pod "e4469a92-c400-468a-8309-13f96c4509dc" (UID: "e4469a92-c400-468a-8309-13f96c4509dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:21.408248 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.408117    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-certs" pod "e4469a92-c400-468a-8309-13f96c4509dc" (UID: "e4469a92-c400-468a-8309-13f96c4509dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:21.408514 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.408467    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4469a92-c400-468a-8309-13f96c4509dc-istiod-ca-cert" pod "e4469a92-c400-468a-8309-13f96c4509dc" (UID: "e4469a92-c400-468a-8309-13f96c4509dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:21.414850 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.414831    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-envoy" pod "e4469a92-c400-468a-8309-13f96c4509dc" (UID: "e4469a92-c400-468a-8309-13f96c4509dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:21.414983 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.414889    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-kube-api-access-wgz2f" pod "e4469a92-c400-468a-8309-13f96c4509dc" (UID: "e4469a92-c400-468a-8309-13f96c4509dc"). InnerVolumeSpecName "kube-api-access-wgz2f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:21.415084 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.415041    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4469a92-c400-468a-8309-13f96c4509dc-istio-podinfo" pod "e4469a92-c400-468a-8309-13f96c4509dc" (UID: "e4469a92-c400-468a-8309-13f96c4509dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:15:21.415352 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.415293    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-istio-token" pod "e4469a92-c400-468a-8309-13f96c4509dc" (UID: "e4469a92-c400-468a-8309-13f96c4509dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:21.426792 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.426765    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:21.507275 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.507193    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.507275 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.507221    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgz2f\" (UniqueName: \"kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-kube-api-access-wgz2f\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.507275 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.507231    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4469a92-c400-468a-8309-13f96c4509dc-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.507275 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.507245    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.507275 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.507257    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4469a92-c400-468a-8309-13f96c4509dc-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.507275 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.507266    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.507275 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.507275    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.507782 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.507286    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4469a92-c400-468a-8309-13f96c4509dc-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.507782 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.507295    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4469a92-c400-468a-8309-13f96c4509dc-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:21.708282 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.708233    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"]
Oct 30 19:15:21.727686 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.727646    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"]
Oct 30 19:15:21.872228 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:21.872068    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:22.056741 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:22.056655    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:22.273000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:22.272955    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:22.273492 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:22.273461    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh" event={"ID":"e4469a92-c400-468a-8309-13f96c4509dc","Type":"ContainerDied","Data":"220bcf64c4729afc9420ba862d93e3327eceae997be96865868580d665ce895b"}
Oct 30 19:15:22.273681 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:22.273535    4925 scope.go:122] "RemoveContainer" containerID="bc4488f827c10ecf28a470a1d5f333b96968f1cad11abd24e88d28e6871a9ef2"
Oct 30 19:15:22.273681 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:22.273666    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"
Oct 30 19:15:22.317175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:22.317148    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"]
Oct 30 19:15:22.319075 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:22.319059    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-baa504-openshift-default-6944b9c4d5-xtlxh"]
Oct 30 19:15:22.435551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:22.435522    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="547ccaea-7848-46dd-a638-3411f31835f7" path="/var/lib/kubelet/pods/547ccaea-7848-46dd-a638-3411f31835f7/volumes"
Oct 30 19:15:22.436038 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:22.436022    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4469a92-c400-468a-8309-13f96c4509dc" path="/var/lib/kubelet/pods/e4469a92-c400-468a-8309-13f96c4509dc/volumes"
Oct 30 19:15:23.052405 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:23.052234    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:23.542532 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:23.542480    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l" podUID="129fe17f-da6a-40e0-8b8f-1c4764b5bf4c" containerName="payload-processing" containerID="cri-o://525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5" gracePeriod=25
Oct 30 19:15:23.554565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:23.554525    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4" podUID="0fbcbf42-c4b7-4531-a941-2be9622bd9d5" containerName="payload-pre-processing" containerID="cri-o://8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7" gracePeriod=25
Oct 30 19:15:23.627506 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:23.627458    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"]
Oct 30 19:15:23.627759 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:23.627726    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m" podUID="8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76" containerName="maas-api" containerID="cri-o://e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5" gracePeriod=30
Oct 30 19:15:23.685604 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:23.684259    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:23.757770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:23.757737    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"]
Oct 30 19:15:23.763876 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:23.763832    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m" podUID="8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.172:8443/health\": dial tcp 10.132.0.172:8443: connect: connection refused"
Oct 30 19:15:23.775524 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:23.775496    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-extmod-b-28c1e9-hgl5w"]
Oct 30 19:15:23.961871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:23.961819    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:24.130218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.129674    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t" podUID="2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:24.207373 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.207344    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:15:24.224436 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.224406    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-maas-api-tls\") pod \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") "
Oct 30 19:15:24.224638 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.224615    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-openshift-service-ca\") pod \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") "
Oct 30 19:15:24.224830 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.224810    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-kube-api-access-x2bgs\" (UniqueName: \"kubernetes.io/projected/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-kube-api-access-x2bgs\") pod \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") "
Oct 30 19:15:24.224915 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.224868    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-metrics-certs\") pod \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\" (UID: \"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76\") "
Oct 30 19:15:24.224984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.224957    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-openshift-service-ca" pod "8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76" (UID: "8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:24.225531 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.225498    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:24.227030 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.226804    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-maas-api-tls" pod "8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76" (UID: "8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:24.227693 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.227668    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-kube-api-access-x2bgs" pod "8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76" (UID: "8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76"). InnerVolumeSpecName "kube-api-access-x2bgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:24.228523 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.228471    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-metrics-certs" pod "8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76" (UID: "8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:24.305055 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.305019    4925 generic.go:356] "Generic (PLEG): container finished" podID="8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76" containerID="e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5" exitCode=0
Oct 30 19:15:24.305297 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.305080    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"
Oct 30 19:15:24.305297 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.305151    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m" event={"ID":"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76","Type":"ContainerDied","Data":"e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5"}
Oct 30 19:15:24.305297 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.305235    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m" event={"ID":"8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76","Type":"ContainerDied","Data":"8bca601faa9c2e855655a7f72c6feddf91dc32af697ca6a66db910fb50217aa3"}
Oct 30 19:15:24.305297 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.305269    4925 scope.go:122] "RemoveContainer" containerID="e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5"
Oct 30 19:15:24.319720 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.319700    4925 scope.go:122] "RemoveContainer" containerID="e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5"
Oct 30 19:15:24.320065 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:24.320041    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5\": container with ID starting with e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5 not found: ID does not exist" containerID="e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5"
Oct 30 19:15:24.320187 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.320078    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5"} err="failed to get container status \"e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5\": rpc error: code = NotFound desc = could not find container \"e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5\": container with ID starting with e4a5ac62a08ba15b68a7ffabc964c4bbc8450035cc687035515e63502d14cac5 not found: ID does not exist"
Oct 30 19:15:24.326427 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.326405    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:24.326427 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.326427    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:24.326541 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.326438    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2bgs\" (UniqueName: \"kubernetes.io/projected/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76-kube-api-access-x2bgs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:24.333908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.333884    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"]
Oct 30 19:15:24.338028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.338006    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-ef6b56-5cc4dcc69c-wlv7m"]
Oct 30 19:15:24.348644 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.348618    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:24.499378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.497838    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14fb2633-bb43-4ae3-8ab4-b21e2011df09" path="/var/lib/kubelet/pods/14fb2633-bb43-4ae3-8ab4-b21e2011df09/volumes"
Oct 30 19:15:24.499378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.498667    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76" path="/var/lib/kubelet/pods/8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76/volumes"
Oct 30 19:15:24.537393 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.537364    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:24.917664 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:24.917630    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:25.306957 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:25.306874    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:26.009842 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:26.009807    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:26.470148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:26.470113    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"]
Oct 30 19:15:26.620027 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:26.619991    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:26.715602 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:26.714276    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx" podUID="86f2d7f0-ab22-4a23-81c7-63536b3fb2ad" containerName="payload-processing" containerID="cri-o://c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892" gracePeriod=25
Oct 30 19:15:26.733996 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:26.733717    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd" podUID="6c157a5a-f699-4ea7-a5d7-7b4bb851ef88" containerName="payload-pre-processing" containerID="cri-o://18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4" gracePeriod=25
Oct 30 19:15:26.821033 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:26.821000    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:27.460760 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:27.457127    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:28.089412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:28.086611    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:28.485211 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:28.485170    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4" podUID="0fbcbf42-c4b7-4531-a941-2be9622bd9d5" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:28.485685 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:28.485345    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l" podUID="129fe17f-da6a-40e0-8b8f-1c4764b5bf4c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:28.621354 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:28.621319    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd" podUID="6c157a5a-f699-4ea7-a5d7-7b4bb851ef88" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:28.856644 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:28.855777    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:29.463607 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:29.463552    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:29.627859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:29.627815    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx" podUID="86f2d7f0-ab22-4a23-81c7-63536b3fb2ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:29.859136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:29.859004    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"]
Oct 30 19:15:29.859329 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:29.859296    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4" podUID="3b55892d-8b94-4401-bc08-65bd855471f5" containerName="maas-api" containerID="cri-o://436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a" gracePeriod=30
Oct 30 19:15:30.021811 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.021767    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:30.101512 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.101444    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"]
Oct 30 19:15:30.108240 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.108218    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w0-68b466-rv7st"]
Oct 30 19:15:30.245124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.242790    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:30.356123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.355380    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:15:30.378848 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.378751    4925 generic.go:356] "Generic (PLEG): container finished" podID="3b55892d-8b94-4401-bc08-65bd855471f5" containerID="436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a" exitCode=0
Oct 30 19:15:30.379158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.379083    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4" event={"ID":"3b55892d-8b94-4401-bc08-65bd855471f5","Type":"ContainerDied","Data":"436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a"}
Oct 30 19:15:30.379327 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.379312    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4" event={"ID":"3b55892d-8b94-4401-bc08-65bd855471f5","Type":"ContainerDied","Data":"5fd55842d9ab70ea4f555492db414013e041f221300ad0dd82270c6f10db34f3"}
Oct 30 19:15:30.379436 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.379425    4925 scope.go:122] "RemoveContainer" containerID="436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a"
Oct 30 19:15:30.379770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.379751    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"
Oct 30 19:15:30.486837 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.486539    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b55892d-8b94-4401-bc08-65bd855471f5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b55892d-8b94-4401-bc08-65bd855471f5-openshift-service-ca\") pod \"3b55892d-8b94-4401-bc08-65bd855471f5\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") "
Oct 30 19:15:30.493001 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.492973    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-maas-api-tls\") pod \"3b55892d-8b94-4401-bc08-65bd855471f5\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") "
Oct 30 19:15:30.528542 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.528511    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b55892d-8b94-4401-bc08-65bd855471f5-kube-api-access-n677c\" (UniqueName: \"kubernetes.io/projected/3b55892d-8b94-4401-bc08-65bd855471f5-kube-api-access-n677c\") pod \"3b55892d-8b94-4401-bc08-65bd855471f5\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") "
Oct 30 19:15:30.528939 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.528910    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-metrics-certs\") pod \"3b55892d-8b94-4401-bc08-65bd855471f5\" (UID: \"3b55892d-8b94-4401-bc08-65bd855471f5\") "
Oct 30 19:15:30.532566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.500638    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964" path="/var/lib/kubelet/pods/11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964/volumes"
Oct 30 19:15:30.532566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.500288    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b55892d-8b94-4401-bc08-65bd855471f5-openshift-service-ca" pod "3b55892d-8b94-4401-bc08-65bd855471f5" (UID: "3b55892d-8b94-4401-bc08-65bd855471f5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:30.532566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.527659    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-maas-api-tls" pod "3b55892d-8b94-4401-bc08-65bd855471f5" (UID: "3b55892d-8b94-4401-bc08-65bd855471f5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:30.533510 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.533492    4925 scope.go:122] "RemoveContainer" containerID="436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a"
Oct 30 19:15:30.533735 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.533525    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b55892d-8b94-4401-bc08-65bd855471f5-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:30.533735 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.533729    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:30.534284 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:30.534049    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a\": container with ID starting with 436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a not found: ID does not exist" containerID="436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a"
Oct 30 19:15:30.534284 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.534239    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a"} err="failed to get container status \"436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a\": rpc error: code = NotFound desc = could not find container \"436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a\": container with ID starting with 436fbb8ff722e3147c6b193b5b54fe066d56b29e88fc8db5be05ce2c58e1c38a not found: ID does not exist"
Oct 30 19:15:30.540514 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.540492    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-metrics-certs" pod "3b55892d-8b94-4401-bc08-65bd855471f5" (UID: "3b55892d-8b94-4401-bc08-65bd855471f5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:30.540683 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.540661    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b55892d-8b94-4401-bc08-65bd855471f5-kube-api-access-n677c" pod "3b55892d-8b94-4401-bc08-65bd855471f5" (UID: "3b55892d-8b94-4401-bc08-65bd855471f5"). InnerVolumeSpecName "kube-api-access-n677c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:30.621311 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.621274    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:30.635675 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.635646    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n677c\" (UniqueName: \"kubernetes.io/projected/3b55892d-8b94-4401-bc08-65bd855471f5-kube-api-access-n677c\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:30.635675 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.635672    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b55892d-8b94-4401-bc08-65bd855471f5-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:30.719432 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.719401    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"]
Oct 30 19:15:30.720857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.720784    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-68b466-6b85b9d788-hpvj4"]
Oct 30 19:15:30.796394 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:30.796296    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:31.068668 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:31.068584    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:31.314170 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:31.314115    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:31.471917 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:31.471872    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn" podUID="a98e91b2-25b6-4853-832f-e912c92ea068" containerName="payload-pre-processing" containerID="cri-o://67229149ece0547c92f551084f91a7c81b2b9a5cf4811701b75cf2aa8a2ad1c7" gracePeriod=25
Oct 30 19:15:31.739260 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:31.732305    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:32.111414 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:32.111320    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:32.297781 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:32.297746    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:32.349276 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:32.349242    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"]
Oct 30 19:15:32.349542 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:32.349518    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy" containerID="cri-o://27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03" gracePeriod=30
Oct 30 19:15:32.392139 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:32.392043    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"]
Oct 30 19:15:32.392318 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:32.392291    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy" containerID="cri-o://5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed" gracePeriod=30
Oct 30 19:15:32.437757 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:32.437725    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b55892d-8b94-4401-bc08-65bd855471f5" path="/var/lib/kubelet/pods/3b55892d-8b94-4401-bc08-65bd855471f5/volumes"
Oct 30 19:15:32.487745 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:32.487713    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:32.685077 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:32.685041    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:33.461575 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:33.461535    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn" podUID="a98e91b2-25b6-4853-832f-e912c92ea068" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:33.669174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:33.667608    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:34.029552 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:34.029516    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t" podUID="2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:34.538124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:34.537714    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:35.178834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:35.178247    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:35.815630 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:35.815588    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:36.580200 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:36.577713    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:37.457443 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.457401    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:37.473673 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.473410    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:15:37.473673 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.473488    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 19:15:37.729677 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.729256    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:15:37.785199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.785172    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:15:37.868991 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.868941    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-socket\") pod \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") "
Oct 30 19:15:37.871044 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.869305    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-credential-socket\") pod \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") "
Oct 30 19:15:37.871276 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.871254    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-certs\") pod \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") "
Oct 30 19:15:37.871472 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.871455    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-token\" (UniqueName: \"kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-token\") pod \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") "
Oct 30 19:15:37.871808 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.871786    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-envoy\") pod \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") "
Oct 30 19:15:37.871978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.871961    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istiod-ca-cert\") pod \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") "
Oct 30 19:15:37.872231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.872210    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-podinfo\") pod \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") "
Oct 30 19:15:37.872442 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.872424    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-kube-api-access-sndz7\" (UniqueName: \"kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-kube-api-access-sndz7\") pod \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") "
Oct 30 19:15:37.872651 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.872633    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-data\") pod \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\" (UID: \"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8\") "
Oct 30 19:15:37.873905 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.873884    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-data" pod "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" (UID: "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:37.874295 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.874272    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-socket" pod "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" (UID: "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:37.874638 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.874619    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-credential-socket" pod "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" (UID: "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:37.875059 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.875039    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-certs" pod "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" (UID: "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:37.879349 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.879319    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istiod-ca-cert" pod "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" (UID: "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:37.881607 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.881518    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:37.886610 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.886572    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-envoy" pod "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" (UID: "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:37.887320 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.887286    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-kube-api-access-sndz7" pod "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" (UID: "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8"). InnerVolumeSpecName "kube-api-access-sndz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:37.887635 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.887598    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-podinfo" pod "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" (UID: "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:15:37.888861 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.888837    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-token" pod "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" (UID: "a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:37.974292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.974222    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-kube-api-access-488bh\" (UniqueName: \"kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-kube-api-access-488bh\") pod \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") "
Oct 30 19:15:37.974396 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.974304    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-credential-socket\") pod \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") "
Oct 30 19:15:37.974396 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.974335    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-envoy\") pod \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") "
Oct 30 19:15:37.974489 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.974408    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istiod-ca-cert\") pod \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") "
Oct 30 19:15:37.974489 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.974449    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-podinfo\") pod \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") "
Oct 30 19:15:37.974489 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.974469    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-certs\") pod \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") "
Oct 30 19:15:37.974703 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.974532    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-token\" (UniqueName: \"kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-token\") pod \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") "
Oct 30 19:15:37.974703 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.974560    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-socket\") pod \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") "
Oct 30 19:15:37.974703 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.974602    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-data\") pod \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\" (UID: \"ae0e1c00-9ab0-4e0b-911b-31e1843d9232\") "
Oct 30 19:15:37.974703 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.974642    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-credential-socket" pod "ae0e1c00-9ab0-4e0b-911b-31e1843d9232" (UID: "ae0e1c00-9ab0-4e0b-911b-31e1843d9232"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:37.974936 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.974854    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istiod-ca-cert" pod "ae0e1c00-9ab0-4e0b-911b-31e1843d9232" (UID: "ae0e1c00-9ab0-4e0b-911b-31e1843d9232"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:37.975072 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975047    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-data" pod "ae0e1c00-9ab0-4e0b-911b-31e1843d9232" (UID: "ae0e1c00-9ab0-4e0b-911b-31e1843d9232"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:37.975441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975414    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-certs" pod "ae0e1c00-9ab0-4e0b-911b-31e1843d9232" (UID: "ae0e1c00-9ab0-4e0b-911b-31e1843d9232"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:37.975441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.974945    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-socket" pod "ae0e1c00-9ab0-4e0b-911b-31e1843d9232" (UID: "ae0e1c00-9ab0-4e0b-911b-31e1843d9232"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:37.975611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975479    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:37.975611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975505    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sndz7\" (UniqueName: \"kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-kube-api-access-sndz7\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:37.975611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975535    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:37.975611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975550    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:37.975611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975565    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:37.975611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975580    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:37.975611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975594    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:37.975611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975608    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:37.975938 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975622    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:37.975938 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975636    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:37.975938 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.975652    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:37.976634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.976601    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-kube-api-access-488bh" pod "ae0e1c00-9ab0-4e0b-911b-31e1843d9232" (UID: "ae0e1c00-9ab0-4e0b-911b-31e1843d9232"). InnerVolumeSpecName "kube-api-access-488bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:37.977204 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.977182    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-token" pod "ae0e1c00-9ab0-4e0b-911b-31e1843d9232" (UID: "ae0e1c00-9ab0-4e0b-911b-31e1843d9232"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:37.977295 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.977270    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-podinfo" pod "ae0e1c00-9ab0-4e0b-911b-31e1843d9232" (UID: "ae0e1c00-9ab0-4e0b-911b-31e1843d9232"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:15:37.977541 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:37.977521    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-envoy" pod "ae0e1c00-9ab0-4e0b-911b-31e1843d9232" (UID: "ae0e1c00-9ab0-4e0b-911b-31e1843d9232"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:38.064068 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.064034    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"]
Oct 30 19:15:38.064691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.064342    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerName="istio-proxy" containerID="cri-o://f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf" gracePeriod=30
Oct 30 19:15:38.087149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.076855    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"]
Oct 30 19:15:38.087149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.077087    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" podUID="7495850d-1b3d-4df3-8424-ebe372d9eb88" containerName="istio-proxy" containerID="cri-o://2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697" gracePeriod=30
Oct 30 19:15:38.087149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.080675    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-488bh\" (UniqueName: \"kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-kube-api-access-488bh\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:38.087149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.080705    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:38.087149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.080719    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:38.087149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.080733    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:38.087149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.080747    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:38.087149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.080761    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:38.087149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.080774    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae0e1c00-9ab0-4e0b-911b-31e1843d9232-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:38.090759 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.090730    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:38.298202 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.297495    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:38.483723 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.483689    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4" podUID="0fbcbf42-c4b7-4531-a941-2be9622bd9d5" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:38.483723 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.483704    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l" podUID="129fe17f-da6a-40e0-8b8f-1c4764b5bf4c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:38.485503 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.485478    4925 generic.go:356] "Generic (PLEG): container finished" podID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerID="27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03" exitCode=0
Oct 30 19:15:38.485646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.485543    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"
Oct 30 19:15:38.485646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.485567    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" event={"ID":"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8","Type":"ContainerDied","Data":"27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03"}
Oct 30 19:15:38.485646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.485633    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt" event={"ID":"a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8","Type":"ContainerDied","Data":"7da4a73f9d59e1029f4359024f7fcc11c8099a6caec8c17197c655df7c39a38d"}
Oct 30 19:15:38.485828 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.485663    4925 scope.go:122] "RemoveContainer" containerID="27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03"
Oct 30 19:15:38.487772 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.487749    4925 generic.go:356] "Generic (PLEG): container finished" podID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerID="5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed" exitCode=0
Oct 30 19:15:38.487945 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.487843    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"
Oct 30 19:15:38.489606 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.489568    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" event={"ID":"ae0e1c00-9ab0-4e0b-911b-31e1843d9232","Type":"ContainerDied","Data":"5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed"}
Oct 30 19:15:38.489729 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.489629    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf" event={"ID":"ae0e1c00-9ab0-4e0b-911b-31e1843d9232","Type":"ContainerDied","Data":"a6a982921fb216892614e4d3958e6c30183fddc3a986f594b82936010aee0ab5"}
Oct 30 19:15:38.498239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.498216    4925 scope.go:122] "RemoveContainer" containerID="c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634"
Oct 30 19:15:38.508729 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.508557    4925 scope.go:122] "RemoveContainer" containerID="27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03"
Oct 30 19:15:38.508859 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:38.508841    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03\": container with ID starting with 27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03 not found: ID does not exist" containerID="27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03"
Oct 30 19:15:38.508904 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.508867    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03"} err="failed to get container status \"27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03\": rpc error: code = NotFound desc = could not find container \"27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03\": container with ID starting with 27ddf1669e7adfee8bab020e3523fa172bc7a51d6c98570b29e5337ad5affd03 not found: ID does not exist"
Oct 30 19:15:38.508904 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.508882    4925 scope.go:122] "RemoveContainer" containerID="c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634"
Oct 30 19:15:38.509162 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:38.509140    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634\": container with ID starting with c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634 not found: ID does not exist" containerID="c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634"
Oct 30 19:15:38.509220 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.509170    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634"} err="failed to get container status \"c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634\": rpc error: code = NotFound desc = could not find container \"c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634\": container with ID starting with c6b30a05f7137cd21acff4f950fb35421c4b1b7054b4e8e3506a1dfce6a22634 not found: ID does not exist"
Oct 30 19:15:38.509220 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.509185    4925 scope.go:122] "RemoveContainer" containerID="5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed"
Oct 30 19:15:38.518727 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.518701    4925 scope.go:122] "RemoveContainer" containerID="45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0"
Oct 30 19:15:38.528777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.528759    4925 scope.go:122] "RemoveContainer" containerID="5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed"
Oct 30 19:15:38.529055 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:38.529036    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed\": container with ID starting with 5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed not found: ID does not exist" containerID="5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed"
Oct 30 19:15:38.529172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.529061    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed"} err="failed to get container status \"5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed\": rpc error: code = NotFound desc = could not find container \"5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed\": container with ID starting with 5f5f985b5ab04d6f0d2be5d178b134309d8a0ab481bb3cde52864aad9fecb4ed not found: ID does not exist"
Oct 30 19:15:38.529172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.529076    4925 scope.go:122] "RemoveContainer" containerID="45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0"
Oct 30 19:15:38.529356 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:38.529336    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0\": container with ID starting with 45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0 not found: ID does not exist" containerID="45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0"
Oct 30 19:15:38.529412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.529366    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0"} err="failed to get container status \"45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0\": rpc error: code = NotFound desc = could not find container \"45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0\": container with ID starting with 45bb0a01c05a638a3ac8ec6a8e25ffdfe4b9b5e358efd900c028236d054d81a0 not found: ID does not exist"
Oct 30 19:15:38.545431 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.545385    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"]
Oct 30 19:15:38.560352 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.560295    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-ef6b56-openshift-default-768647479-wp2lf"]
Oct 30 19:15:38.592047 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.592018    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"]
Oct 30 19:15:38.604737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.604710    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-28c1e9-openshift-default-6cf9f44bc7-kqjmt"]
Oct 30 19:15:38.620441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.620414    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd" podUID="6c157a5a-f699-4ea7-a5d7-7b4bb851ef88" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:38.741169 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:38.741087    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:39.386401 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:39.385764    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:39.626740 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:39.626697    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx" podUID="86f2d7f0-ab22-4a23-81c7-63536b3fb2ad" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:40.288303 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:40.285243    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:40.425274 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:40.425246    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" path="/var/lib/kubelet/pods/a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8/volumes"
Oct 30 19:15:40.425857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:40.425841    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" path="/var/lib/kubelet/pods/ae0e1c00-9ab0-4e0b-911b-31e1843d9232/volumes"
Oct 30 19:15:40.901815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:40.901722    4925 patch_prober.go:29] interesting pod/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:15:40.902345 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:40.902317    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:15:40.965997 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:40.965966    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"
Oct 30 19:15:41.082079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.082035    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-kube-api-access-lwq9f\" (UniqueName: \"kubernetes.io/projected/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-kube-api-access-lwq9f\") pod \"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab\" (UID: \"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab\") "
Oct 30 19:15:41.082294 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.082177    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-plugins-config-volume\") pod \"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab\" (UID: \"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab\") "
Oct 30 19:15:41.083041 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.083002    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:41.088753 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.087198    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-plugins-config-volume" pod "2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab" (UID: "2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:41.095181 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.093872    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-kube-api-access-lwq9f" pod "2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab" (UID: "2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab"). InnerVolumeSpecName "kube-api-access-lwq9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:41.184201 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.184173    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwq9f\" (UniqueName: \"kubernetes.io/projected/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-kube-api-access-lwq9f\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:41.184201 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.184198    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:41.515184 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.515068    4925 generic.go:356] "Generic (PLEG): container finished" podID="2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab" containerID="15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f" exitCode=0
Oct 30 19:15:41.515184 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.515132    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"
Oct 30 19:15:41.515184 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.515157    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t" event={"ID":"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab","Type":"ContainerDied","Data":"15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f"}
Oct 30 19:15:41.515488 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.515224    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t" event={"ID":"2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab","Type":"ContainerDied","Data":"28fa77795b555172cad5c475d803aac60a7be0507c945fc36f03267d517813c4"}
Oct 30 19:15:41.515488 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.515257    4925 scope.go:122] "RemoveContainer" containerID="15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f"
Oct 30 19:15:41.529051 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.529022    4925 scope.go:122] "RemoveContainer" containerID="15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f"
Oct 30 19:15:41.529411 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:41.529383    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f\": container with ID starting with 15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f not found: ID does not exist" containerID="15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f"
Oct 30 19:15:41.529487 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.529424    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f"} err="failed to get container status \"15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f\": rpc error: code = NotFound desc = could not find container \"15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f\": container with ID starting with 15c751ab345fa661a0bc09d6a8596bb267fc3ab20026753b779a967301867d3f not found: ID does not exist"
Oct 30 19:15:41.542577 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.542555    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"]
Oct 30 19:15:41.546305 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.546285    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-d6f956-664c46f69b-hzl8t"]
Oct 30 19:15:41.685120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:41.682673    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:42.339732 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:42.339085    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:42.423402 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:42.423362    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab" path="/var/lib/kubelet/pods/2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab/volumes"
Oct 30 19:15:43.058462 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.040182    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"]
Oct 30 19:15:43.075349 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.073920    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"]
Oct 30 19:15:43.079268 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.076831    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:43.274465 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.273918    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:43.322206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.318189    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"]
Oct 30 19:15:43.322206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.320901    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"]
Oct 30 19:15:43.329657 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:43.329415    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:15:43.488719 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.488686    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:43.495140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.493937    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn" podUID="a98e91b2-25b6-4853-832f-e912c92ea068" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:43.509676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.509466    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:15:43.512866 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.512835    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:15:43.542496 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.542455    4925 generic.go:356] "Generic (PLEG): container finished" podID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerID="f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf" exitCode=0
Oct 30 19:15:43.542722 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.542705    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"
Oct 30 19:15:43.542889 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.542863    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" event={"ID":"5e69194e-043b-48dc-aa47-4f6d378a09a4","Type":"ContainerDied","Data":"f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf"}
Oct 30 19:15:43.542950 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.542934    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z" event={"ID":"5e69194e-043b-48dc-aa47-4f6d378a09a4","Type":"ContainerDied","Data":"1808e0165d2151be08be2da0c0cef823d1f572e0de0496803949539881d23741"}
Oct 30 19:15:43.542995 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.542976    4925 scope.go:122] "RemoveContainer" containerID="f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf"
Oct 30 19:15:43.545699 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.545671    4925 generic.go:356] "Generic (PLEG): container finished" podID="7495850d-1b3d-4df3-8424-ebe372d9eb88" containerID="2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697" exitCode=0
Oct 30 19:15:43.545835 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.545817    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" event={"ID":"7495850d-1b3d-4df3-8424-ebe372d9eb88","Type":"ContainerDied","Data":"2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697"}
Oct 30 19:15:43.545897 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.545883    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw" event={"ID":"7495850d-1b3d-4df3-8424-ebe372d9eb88","Type":"ContainerDied","Data":"3c005a400bf205749057bda21abb0d5386d52450f40ad37fdd3338ad6aee9ff0"}
Oct 30 19:15:43.545993 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.545978    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.566705    4925 scope.go:122] "RemoveContainer" containerID="f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf"
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:43.567089    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf\": container with ID starting with f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf not found: ID does not exist" containerID="f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf"
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.567146    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf"} err="failed to get container status \"f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf\": rpc error: code = NotFound desc = could not find container \"f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf\": container with ID starting with f778bc89ab0287efcb2ce4f27378cc40226fac23a7b2c262a43fc9a24a766edf not found: ID does not exist"
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.567170    4925 scope.go:122] "RemoveContainer" containerID="2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697"
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.588938    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-envoy\") pod \"5e69194e-043b-48dc-aa47-4f6d378a09a4\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.589006    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-socket\") pod \"7495850d-1b3d-4df3-8424-ebe372d9eb88\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.589061    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-kube-api-access-ct72d\" (UniqueName: \"kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-kube-api-access-ct72d\") pod \"7495850d-1b3d-4df3-8424-ebe372d9eb88\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.589472    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-socket" pod "7495850d-1b3d-4df3-8424-ebe372d9eb88" (UID: "7495850d-1b3d-4df3-8424-ebe372d9eb88"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.589811    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-envoy\") pod \"7495850d-1b3d-4df3-8424-ebe372d9eb88\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.589919    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-certs\") pod \"5e69194e-043b-48dc-aa47-4f6d378a09a4\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.589998    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-podinfo\") pod \"5e69194e-043b-48dc-aa47-4f6d378a09a4\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.590037    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-token\" (UniqueName: \"kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-token\") pod \"7495850d-1b3d-4df3-8424-ebe372d9eb88\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.590087    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-token\" (UniqueName: \"kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-token\") pod \"5e69194e-043b-48dc-aa47-4f6d378a09a4\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.590180    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-kube-api-access-t9v54\" (UniqueName: \"kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-kube-api-access-t9v54\") pod \"5e69194e-043b-48dc-aa47-4f6d378a09a4\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.590216    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-socket\") pod \"5e69194e-043b-48dc-aa47-4f6d378a09a4\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.590265    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7495850d-1b3d-4df3-8424-ebe372d9eb88-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7495850d-1b3d-4df3-8424-ebe372d9eb88-istiod-ca-cert\") pod \"7495850d-1b3d-4df3-8424-ebe372d9eb88\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.590303    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-certs\") pod \"7495850d-1b3d-4df3-8424-ebe372d9eb88\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.590384    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-data\") pod \"5e69194e-043b-48dc-aa47-4f6d378a09a4\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.590429    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-data\") pod \"7495850d-1b3d-4df3-8424-ebe372d9eb88\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.590484    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-credential-socket\") pod \"5e69194e-043b-48dc-aa47-4f6d378a09a4\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.590529    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e69194e-043b-48dc-aa47-4f6d378a09a4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e69194e-043b-48dc-aa47-4f6d378a09a4-istiod-ca-cert\") pod \"5e69194e-043b-48dc-aa47-4f6d378a09a4\" (UID: \"5e69194e-043b-48dc-aa47-4f6d378a09a4\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.590556    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-podinfo\") pod \"7495850d-1b3d-4df3-8424-ebe372d9eb88\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.590594    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-credential-socket\") pod \"7495850d-1b3d-4df3-8424-ebe372d9eb88\" (UID: \"7495850d-1b3d-4df3-8424-ebe372d9eb88\") "
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.593504    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-data" pod "7495850d-1b3d-4df3-8424-ebe372d9eb88" (UID: "7495850d-1b3d-4df3-8424-ebe372d9eb88"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.593796    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-data" pod "5e69194e-043b-48dc-aa47-4f6d378a09a4" (UID: "5e69194e-043b-48dc-aa47-4f6d378a09a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.594542    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-certs" pod "7495850d-1b3d-4df3-8424-ebe372d9eb88" (UID: "7495850d-1b3d-4df3-8424-ebe372d9eb88"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.594873    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7495850d-1b3d-4df3-8424-ebe372d9eb88-istiod-ca-cert" pod "7495850d-1b3d-4df3-8424-ebe372d9eb88" (UID: "7495850d-1b3d-4df3-8424-ebe372d9eb88"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.595438    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-socket" pod "5e69194e-043b-48dc-aa47-4f6d378a09a4" (UID: "5e69194e-043b-48dc-aa47-4f6d378a09a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.596321    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-credential-socket" pod "5e69194e-043b-48dc-aa47-4f6d378a09a4" (UID: "5e69194e-043b-48dc-aa47-4f6d378a09a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:43.606172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.596645    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e69194e-043b-48dc-aa47-4f6d378a09a4-istiod-ca-cert" pod "5e69194e-043b-48dc-aa47-4f6d378a09a4" (UID: "5e69194e-043b-48dc-aa47-4f6d378a09a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:43.645129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.617758    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-credential-socket" pod "7495850d-1b3d-4df3-8424-ebe372d9eb88" (UID: "7495850d-1b3d-4df3-8424-ebe372d9eb88"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:43.645129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.633527    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.645129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.633559    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.645129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.633575    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7495850d-1b3d-4df3-8424-ebe372d9eb88-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.645129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.633589    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.645129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.633602    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.645129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.633617    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.645129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.633631    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.645129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.633644    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e69194e-043b-48dc-aa47-4f6d378a09a4-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.645129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.633657    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.700883 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.700738    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:43.735599 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.713231    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-certs" pod "5e69194e-043b-48dc-aa47-4f6d378a09a4" (UID: "5e69194e-043b-48dc-aa47-4f6d378a09a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:43.801283 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.742389    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.801283 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.772158    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"]
Oct 30 19:15:43.821217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.820605    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-kube-api-access-ct72d" pod "7495850d-1b3d-4df3-8424-ebe372d9eb88" (UID: "7495850d-1b3d-4df3-8424-ebe372d9eb88"). InnerVolumeSpecName "kube-api-access-ct72d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:43.826374 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.826339    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"]
Oct 30 19:15:43.827590 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.827545    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-envoy" pod "7495850d-1b3d-4df3-8424-ebe372d9eb88" (UID: "7495850d-1b3d-4df3-8424-ebe372d9eb88"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:43.849792 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.849742    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ct72d\" (UniqueName: \"kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-kube-api-access-ct72d\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.850386 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.850359    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.850630 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.850605    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-podinfo" pod "7495850d-1b3d-4df3-8424-ebe372d9eb88" (UID: "7495850d-1b3d-4df3-8424-ebe372d9eb88"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:15:43.851392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.851180    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-kube-api-access-t9v54" pod "5e69194e-043b-48dc-aa47-4f6d378a09a4" (UID: "5e69194e-043b-48dc-aa47-4f6d378a09a4"). InnerVolumeSpecName "kube-api-access-t9v54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:43.855533 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.854279    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-token" pod "7495850d-1b3d-4df3-8424-ebe372d9eb88" (UID: "7495850d-1b3d-4df3-8424-ebe372d9eb88"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:43.864086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.864061    4925 scope.go:122] "RemoveContainer" containerID="2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697"
Oct 30 19:15:43.867274 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.867196    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-envoy" pod "5e69194e-043b-48dc-aa47-4f6d378a09a4" (UID: "5e69194e-043b-48dc-aa47-4f6d378a09a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:43.874843 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:43.873607    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697\": container with ID starting with 2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697 not found: ID does not exist" containerID="2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697"
Oct 30 19:15:43.874843 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.873648    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697"} err="failed to get container status \"2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697\": rpc error: code = NotFound desc = could not find container \"2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697\": container with ID starting with 2aa16e4b94e3b2c820f901b9a472cb0af312c63260313d55235038b6b7213697 not found: ID does not exist"
Oct 30 19:15:43.874843 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.873987    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-podinfo" pod "5e69194e-043b-48dc-aa47-4f6d378a09a4" (UID: "5e69194e-043b-48dc-aa47-4f6d378a09a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:15:43.874843 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.874147    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-token" pod "5e69194e-043b-48dc-aa47-4f6d378a09a4" (UID: "5e69194e-043b-48dc-aa47-4f6d378a09a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:43.884404 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.884262    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:43.952151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.951646    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.952151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.951675    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.952151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.951684    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.952151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.951694    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7495850d-1b3d-4df3-8424-ebe372d9eb88-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.952151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.951704    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:43.952151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:43.951714    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9v54\" (UniqueName: \"kubernetes.io/projected/5e69194e-043b-48dc-aa47-4f6d378a09a4-kube-api-access-t9v54\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:44.076134 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.068342    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:44.185826 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.185801    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"
Oct 30 19:15:44.201011 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.200985    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"]
Oct 30 19:15:44.204003 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.203979    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-gnb5z"]
Oct 30 19:15:44.213593 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.213568    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"]
Oct 30 19:15:44.218203 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.218182    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-d6f956-openshift-default-56888545df-c8mzw"]
Oct 30 19:15:44.220678 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.220658    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"
Oct 30 19:15:44.258676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.258026    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-plugins-config-volume\") pod \"0fbcbf42-c4b7-4531-a941-2be9622bd9d5\" (UID: \"0fbcbf42-c4b7-4531-a941-2be9622bd9d5\") "
Oct 30 19:15:44.258676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.258303    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-plugins-config-volume\") pod \"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c\" (UID: \"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c\") "
Oct 30 19:15:44.258676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.258372    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-kube-api-access-x4nwl\" (UniqueName: \"kubernetes.io/projected/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-kube-api-access-x4nwl\") pod \"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c\" (UID: \"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c\") "
Oct 30 19:15:44.259585 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.259333    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-plugins-config-volume" pod "0fbcbf42-c4b7-4531-a941-2be9622bd9d5" (UID: "0fbcbf42-c4b7-4531-a941-2be9622bd9d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:44.259899 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.259861    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-plugins-config-volume" pod "129fe17f-da6a-40e0-8b8f-1c4764b5bf4c" (UID: "129fe17f-da6a-40e0-8b8f-1c4764b5bf4c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:44.260834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.260769    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:44.260834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.260797    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:44.266154 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.266125    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-kube-api-access-x4nwl" pod "129fe17f-da6a-40e0-8b8f-1c4764b5bf4c" (UID: "129fe17f-da6a-40e0-8b8f-1c4764b5bf4c"). InnerVolumeSpecName "kube-api-access-x4nwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:44.361643 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.361615    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-kube-api-access-dbcpb\" (UniqueName: \"kubernetes.io/projected/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-kube-api-access-dbcpb\") pod \"0fbcbf42-c4b7-4531-a941-2be9622bd9d5\" (UID: \"0fbcbf42-c4b7-4531-a941-2be9622bd9d5\") "
Oct 30 19:15:44.362131 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.362082    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4nwl\" (UniqueName: \"kubernetes.io/projected/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c-kube-api-access-x4nwl\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:44.363890 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.363864    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-kube-api-access-dbcpb" pod "0fbcbf42-c4b7-4531-a941-2be9622bd9d5" (UID: "0fbcbf42-c4b7-4531-a941-2be9622bd9d5"). InnerVolumeSpecName "kube-api-access-dbcpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:44.425614 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.425581    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" path="/var/lib/kubelet/pods/5e69194e-043b-48dc-aa47-4f6d378a09a4/volumes"
Oct 30 19:15:44.426315 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.426283    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7495850d-1b3d-4df3-8424-ebe372d9eb88" path="/var/lib/kubelet/pods/7495850d-1b3d-4df3-8424-ebe372d9eb88/volumes"
Oct 30 19:15:44.464057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.464027    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbcpb\" (UniqueName: \"kubernetes.io/projected/0fbcbf42-c4b7-4531-a941-2be9622bd9d5-kube-api-access-dbcpb\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:44.556758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.556723    4925 generic.go:356] "Generic (PLEG): container finished" podID="0fbcbf42-c4b7-4531-a941-2be9622bd9d5" containerID="8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7" exitCode=0
Oct 30 19:15:44.557206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.556791    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"
Oct 30 19:15:44.557206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.556819    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4" event={"ID":"0fbcbf42-c4b7-4531-a941-2be9622bd9d5","Type":"ContainerDied","Data":"8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7"}
Oct 30 19:15:44.557206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.556893    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4" event={"ID":"0fbcbf42-c4b7-4531-a941-2be9622bd9d5","Type":"ContainerDied","Data":"b2d7bb40a8d705968302f80f84abc34e545944b2e4505325c343f06379205491"}
Oct 30 19:15:44.557206 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.556922    4925 scope.go:122] "RemoveContainer" containerID="8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7"
Oct 30 19:15:44.559199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.559174    4925 generic.go:356] "Generic (PLEG): container finished" podID="129fe17f-da6a-40e0-8b8f-1c4764b5bf4c" containerID="525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5" exitCode=0
Oct 30 19:15:44.559347 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.559284    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l" event={"ID":"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c","Type":"ContainerDied","Data":"525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5"}
Oct 30 19:15:44.559446 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.559375    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l" event={"ID":"129fe17f-da6a-40e0-8b8f-1c4764b5bf4c","Type":"ContainerDied","Data":"c78323aab6cb801160ae81ae1dfc3fcdb7cd15e40a526335ed1f3b9858caf970"}
Oct 30 19:15:44.559831 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.559805    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"
Oct 30 19:15:44.571307 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.571282    4925 scope.go:122] "RemoveContainer" containerID="8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7"
Oct 30 19:15:44.571590 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:44.571572    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7\": container with ID starting with 8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7 not found: ID does not exist" containerID="8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7"
Oct 30 19:15:44.571672 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.571600    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7"} err="failed to get container status \"8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7\": rpc error: code = NotFound desc = could not find container \"8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7\": container with ID starting with 8a8e81641a78c17917d745df47d1d62e53b2c73017419069005a4f1ef377bad7 not found: ID does not exist"
Oct 30 19:15:44.571672 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.571622    4925 scope.go:122] "RemoveContainer" containerID="525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5"
Oct 30 19:15:44.581995 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.581972    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"]
Oct 30 19:15:44.585489 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.585465    4925 scope.go:122] "RemoveContainer" containerID="525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5"
Oct 30 19:15:44.585794 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:44.585774    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5\": container with ID starting with 525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5 not found: ID does not exist" containerID="525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5"
Oct 30 19:15:44.585863 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.585799    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5"} err="failed to get container status \"525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5\": rpc error: code = NotFound desc = could not find container \"525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5\": container with ID starting with 525df7bb478a923b7dc7713ce415a2d6cca92831b9113ef5de606b2ab13c18f5 not found: ID does not exist"
Oct 30 19:15:44.587803 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.587782    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-28c1e9-7f799964f5-gfmv4"]
Oct 30 19:15:44.599715 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.599689    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"]
Oct 30 19:15:44.604854 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.604832    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-28c1e9-764f79d546-jml8l"]
Oct 30 19:15:44.695681 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:44.695575    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:45.496497 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:45.496437    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:46.182932 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:46.176798    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:46.423975 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:46.423942    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0fbcbf42-c4b7-4531-a941-2be9622bd9d5" path="/var/lib/kubelet/pods/0fbcbf42-c4b7-4531-a941-2be9622bd9d5/volumes"
Oct 30 19:15:46.424403 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:46.424388    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="129fe17f-da6a-40e0-8b8f-1c4764b5bf4c" path="/var/lib/kubelet/pods/129fe17f-da6a-40e0-8b8f-1c4764b5bf4c/volumes"
Oct 30 19:15:46.762661 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:46.762535    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:46.976262 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:46.976233    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"
Oct 30 19:15:47.019113 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.019075    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"
Oct 30 19:15:47.126044 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.126006    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-kube-api-access-qpznf\" (UniqueName: \"kubernetes.io/projected/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-kube-api-access-qpznf\") pod \"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88\" (UID: \"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88\") "
Oct 30 19:15:47.126266 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.126111    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-plugins-config-volume\") pod \"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88\" (UID: \"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88\") "
Oct 30 19:15:47.126266 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.126180    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-kube-api-access-l8vnd\" (UniqueName: \"kubernetes.io/projected/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-kube-api-access-l8vnd\") pod \"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad\" (UID: \"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad\") "
Oct 30 19:15:47.126266 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.126230    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-plugins-config-volume\") pod \"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad\" (UID: \"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad\") "
Oct 30 19:15:47.126519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.126485    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-plugins-config-volume" pod "6c157a5a-f699-4ea7-a5d7-7b4bb851ef88" (UID: "6c157a5a-f699-4ea7-a5d7-7b4bb851ef88"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:47.126747 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.126727    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-plugins-config-volume" pod "86f2d7f0-ab22-4a23-81c7-63536b3fb2ad" (UID: "86f2d7f0-ab22-4a23-81c7-63536b3fb2ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:47.126924 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.126903    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:47.127002 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.126927    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:47.128419 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.128396    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-kube-api-access-qpznf" pod "6c157a5a-f699-4ea7-a5d7-7b4bb851ef88" (UID: "6c157a5a-f699-4ea7-a5d7-7b4bb851ef88"). InnerVolumeSpecName "kube-api-access-qpznf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:47.128806 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.128793    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-kube-api-access-l8vnd" pod "86f2d7f0-ab22-4a23-81c7-63536b3fb2ad" (UID: "86f2d7f0-ab22-4a23-81c7-63536b3fb2ad"). InnerVolumeSpecName "kube-api-access-l8vnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:47.228271 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.228242    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8vnd\" (UniqueName: \"kubernetes.io/projected/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad-kube-api-access-l8vnd\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:47.228271 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.228267    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpznf\" (UniqueName: \"kubernetes.io/projected/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88-kube-api-access-qpznf\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:47.499216 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.497087    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:47.593912 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.593881    4925 generic.go:356] "Generic (PLEG): container finished" podID="6c157a5a-f699-4ea7-a5d7-7b4bb851ef88" containerID="18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4" exitCode=0
Oct 30 19:15:47.594086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.593939    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"
Oct 30 19:15:47.594086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.593968    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd" event={"ID":"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88","Type":"ContainerDied","Data":"18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4"}
Oct 30 19:15:47.594086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.594022    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd" event={"ID":"6c157a5a-f699-4ea7-a5d7-7b4bb851ef88","Type":"ContainerDied","Data":"af1c1d41000e5620d69e5ea1e8b7d76f2a362a10fe13bf98dff98537556c2ae2"}
Oct 30 19:15:47.594086 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.594054    4925 scope.go:122] "RemoveContainer" containerID="18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4"
Oct 30 19:15:47.596135 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.596111    4925 generic.go:356] "Generic (PLEG): container finished" podID="86f2d7f0-ab22-4a23-81c7-63536b3fb2ad" containerID="c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892" exitCode=0
Oct 30 19:15:47.596272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.596238    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx" event={"ID":"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad","Type":"ContainerDied","Data":"c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892"}
Oct 30 19:15:47.596338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.596275    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx" event={"ID":"86f2d7f0-ab22-4a23-81c7-63536b3fb2ad","Type":"ContainerDied","Data":"d8ba748142a0927ab5003f325bd23c5223f77d64aa55782fca73b705cafff34d"}
Oct 30 19:15:47.596338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.596329    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"
Oct 30 19:15:47.608253 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.608221    4925 scope.go:122] "RemoveContainer" containerID="18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4"
Oct 30 19:15:47.608740 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:47.608718    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4\": container with ID starting with 18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4 not found: ID does not exist" containerID="18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4"
Oct 30 19:15:47.608845 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.608744    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4"} err="failed to get container status \"18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4\": rpc error: code = NotFound desc = could not find container \"18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4\": container with ID starting with 18743e18441a8ac6f65c873c4c2bc5300f23cfb60740e46ddd8d8012899d74e4 not found: ID does not exist"
Oct 30 19:15:47.608845 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.608759    4925 scope.go:122] "RemoveContainer" containerID="c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892"
Oct 30 19:15:47.618973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.618953    4925 scope.go:122] "RemoveContainer" containerID="c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892"
Oct 30 19:15:47.619322 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:47.619303    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892\": container with ID starting with c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892 not found: ID does not exist" containerID="c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892"
Oct 30 19:15:47.619381 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.619335    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892"} err="failed to get container status \"c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892\": rpc error: code = NotFound desc = could not find container \"c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892\": container with ID starting with c17bceb446754e37aa6d3529021d8901c62897f8831c727880ba3a4b8ac7b892 not found: ID does not exist"
Oct 30 19:15:47.624231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.624212    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"]
Oct 30 19:15:47.630243 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.630223    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-ef6b56-db5bbb6bc-bxlwd"]
Oct 30 19:15:47.639945 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.639924    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"]
Oct 30 19:15:47.646154 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:47.646135    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-ef6b56-68548b9ccb-wvwkx"]
Oct 30 19:15:48.041444 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:48.041357    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh" podUID="3107703b-0cfb-4320-84c9-d023eeea6c36" containerName="payload-processing" containerID="cri-o://ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3" gracePeriod=25
Oct 30 19:15:48.074455 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:48.074373    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg" podUID="af4ba1e6-4f4c-42c6-bc34-4eafcc18e428" containerName="payload-pre-processing" containerID="cri-o://b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d" gracePeriod=25
Oct 30 19:15:48.199122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:48.194607    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:48.319291 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:48.319210    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl" podUID="3c79b524-09c4-4e29-8a1c-2306621221db" containerName="payload-processing" containerID="cri-o://2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af" gracePeriod=25
Oct 30 19:15:48.321734 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:48.321703    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv" podUID="2640e252-a00f-462c-bd40-760ba7bc79e0" containerName="payload-pre-processing" containerID="cri-o://52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d" gracePeriod=25
Oct 30 19:15:48.425388 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:48.425356    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c157a5a-f699-4ea7-a5d7-7b4bb851ef88" path="/var/lib/kubelet/pods/6c157a5a-f699-4ea7-a5d7-7b4bb851ef88/volumes"
Oct 30 19:15:48.425887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:48.425871    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86f2d7f0-ab22-4a23-81c7-63536b3fb2ad" path="/var/lib/kubelet/pods/86f2d7f0-ab22-4a23-81c7-63536b3fb2ad/volumes"
Oct 30 19:15:49.029325 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:49.029216    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:49.209890 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:49.209856    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:49.387502 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:49.387038    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:49.964919 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:49.959702    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:50.563862 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:50.563671    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:51.188515 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:51.188482    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:51.239857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:51.239815    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh" podUID="3107703b-0cfb-4320-84c9-d023eeea6c36" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:51.240037 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:51.239833    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg" podUID="af4ba1e6-4f4c-42c6-bc34-4eafcc18e428" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:51.660368 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:51.660331    4925 generic.go:356] "Generic (PLEG): container finished" podID="a98e91b2-25b6-4853-832f-e912c92ea068" containerID="67229149ece0547c92f551084f91a7c81b2b9a5cf4811701b75cf2aa8a2ad1c7" exitCode=0
Oct 30 19:15:51.660953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:51.660929    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn" event={"ID":"a98e91b2-25b6-4853-832f-e912c92ea068","Type":"ContainerDied","Data":"67229149ece0547c92f551084f91a7c81b2b9a5cf4811701b75cf2aa8a2ad1c7"}
Oct 30 19:15:51.888937 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:51.888908    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"
Oct 30 19:15:51.889617 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:51.889588    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a98e91b2-25b6-4853-832f-e912c92ea068-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a98e91b2-25b6-4853-832f-e912c92ea068-plugins-config-volume\") pod \"a98e91b2-25b6-4853-832f-e912c92ea068\" (UID: \"a98e91b2-25b6-4853-832f-e912c92ea068\") "
Oct 30 19:15:51.889777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:51.889667    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a98e91b2-25b6-4853-832f-e912c92ea068-kube-api-access-f8kf9\" (UniqueName: \"kubernetes.io/projected/a98e91b2-25b6-4853-832f-e912c92ea068-kube-api-access-f8kf9\") pod \"a98e91b2-25b6-4853-832f-e912c92ea068\" (UID: \"a98e91b2-25b6-4853-832f-e912c92ea068\") "
Oct 30 19:15:51.890039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:51.890015    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a98e91b2-25b6-4853-832f-e912c92ea068-plugins-config-volume" pod "a98e91b2-25b6-4853-832f-e912c92ea068" (UID: "a98e91b2-25b6-4853-832f-e912c92ea068"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:51.890214 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:51.890195    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a98e91b2-25b6-4853-832f-e912c92ea068-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:51.892164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:51.892137    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a98e91b2-25b6-4853-832f-e912c92ea068-kube-api-access-f8kf9" pod "a98e91b2-25b6-4853-832f-e912c92ea068" (UID: "a98e91b2-25b6-4853-832f-e912c92ea068"). InnerVolumeSpecName "kube-api-access-f8kf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:51.991286 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:51.991257    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8kf9\" (UniqueName: \"kubernetes.io/projected/a98e91b2-25b6-4853-832f-e912c92ea068-kube-api-access-f8kf9\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:52.191120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:52.188562    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:52.579177 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:52.579078    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"]
Oct 30 19:15:52.579630 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:52.579599    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg" podUID="ee31e82e-3665-4a8c-b502-429976b05fd9" containerName="maas-api" containerID="cri-o://d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe" gracePeriod=30
Oct 30 19:15:52.681168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:52.673614    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"
Oct 30 19:15:52.681168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:52.674022    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn" event={"ID":"a98e91b2-25b6-4853-832f-e912c92ea068","Type":"ContainerDied","Data":"fafbd698f7896b7176644bbe3afa4133c2eb67710401fc374a551c684e7d87ac"}
Oct 30 19:15:52.681168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:52.674125    4925 scope.go:122] "RemoveContainer" containerID="67229149ece0547c92f551084f91a7c81b2b9a5cf4811701b75cf2aa8a2ad1c7"
Oct 30 19:15:52.705864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:52.705541    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"]
Oct 30 19:15:52.709462 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:52.709429    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w0-68b466-594d77b4c-nfjqn"]
Oct 30 19:15:52.714551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:52.714521    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:53.007116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.006253    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:53.116280 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.116217    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:15:53.159707 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.159671    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-metrics-certs\") pod \"ee31e82e-3665-4a8c-b502-429976b05fd9\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") "
Oct 30 19:15:53.159910 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.159793    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee31e82e-3665-4a8c-b502-429976b05fd9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee31e82e-3665-4a8c-b502-429976b05fd9-openshift-service-ca\") pod \"ee31e82e-3665-4a8c-b502-429976b05fd9\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") "
Oct 30 19:15:53.159910 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.159863    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee31e82e-3665-4a8c-b502-429976b05fd9-kube-api-access-qbmvx\" (UniqueName: \"kubernetes.io/projected/ee31e82e-3665-4a8c-b502-429976b05fd9-kube-api-access-qbmvx\") pod \"ee31e82e-3665-4a8c-b502-429976b05fd9\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") "
Oct 30 19:15:53.160160 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.159954    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-maas-api-tls\") pod \"ee31e82e-3665-4a8c-b502-429976b05fd9\" (UID: \"ee31e82e-3665-4a8c-b502-429976b05fd9\") "
Oct 30 19:15:53.162868 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.161239    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee31e82e-3665-4a8c-b502-429976b05fd9-openshift-service-ca" pod "ee31e82e-3665-4a8c-b502-429976b05fd9" (UID: "ee31e82e-3665-4a8c-b502-429976b05fd9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:53.163473 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.163445    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-maas-api-tls" pod "ee31e82e-3665-4a8c-b502-429976b05fd9" (UID: "ee31e82e-3665-4a8c-b502-429976b05fd9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:53.170584 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.170546    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-metrics-certs" pod "ee31e82e-3665-4a8c-b502-429976b05fd9" (UID: "ee31e82e-3665-4a8c-b502-429976b05fd9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:53.170584 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.170562    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee31e82e-3665-4a8c-b502-429976b05fd9-kube-api-access-qbmvx" pod "ee31e82e-3665-4a8c-b502-429976b05fd9" (UID: "ee31e82e-3665-4a8c-b502-429976b05fd9"). InnerVolumeSpecName "kube-api-access-qbmvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:53.185022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.184985    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:53.288120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.284757    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee31e82e-3665-4a8c-b502-429976b05fd9-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:53.288120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.284811    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbmvx\" (UniqueName: \"kubernetes.io/projected/ee31e82e-3665-4a8c-b502-429976b05fd9-kube-api-access-qbmvx\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:53.288120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.284833    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:53.288120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.284847    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee31e82e-3665-4a8c-b502-429976b05fd9-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:53.371526 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.371426    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:53.545531 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.545496    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:53.684841 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.684809    4925 generic.go:356] "Generic (PLEG): container finished" podID="ee31e82e-3665-4a8c-b502-429976b05fd9" containerID="d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe" exitCode=0
Oct 30 19:15:53.684841 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.684845    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg" event={"ID":"ee31e82e-3665-4a8c-b502-429976b05fd9","Type":"ContainerDied","Data":"d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe"}
Oct 30 19:15:53.685391 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.684869    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"
Oct 30 19:15:53.685391 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.684882    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg" event={"ID":"ee31e82e-3665-4a8c-b502-429976b05fd9","Type":"ContainerDied","Data":"040d69b26471784271ce26b171510c3bf09c60177376a1424278626f437ef95c"}
Oct 30 19:15:53.685391 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.684904    4925 scope.go:122] "RemoveContainer" containerID="d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe"
Oct 30 19:15:53.697874 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.697854    4925 scope.go:122] "RemoveContainer" containerID="d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe"
Oct 30 19:15:53.698374 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:53.698344    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe\": container with ID starting with d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe not found: ID does not exist" containerID="d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe"
Oct 30 19:15:53.698504 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.698383    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe"} err="failed to get container status \"d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe\": rpc error: code = NotFound desc = could not find container \"d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe\": container with ID starting with d678139c0ebdd7e699baa1b04595ed4a2d5ec6415f6835c4f8d490bd59b1fafe not found: ID does not exist"
Oct 30 19:15:53.711763 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.711741    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"]
Oct 30 19:15:53.717497 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.717470    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c2e3f9-8496d595db-85xzg"]
Oct 30 19:15:53.826166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.819013    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"]
Oct 30 19:15:53.826166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.819274    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr" podUID="0a2211bc-9633-49ee-b65e-7c39f753357d" containerName="maas-api" containerID="cri-o://05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029" gracePeriod=30
Oct 30 19:15:53.934824 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:53.934790    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:54.178133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.172335    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:54.409853 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.409802    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:15:54.427550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.427520    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a98e91b2-25b6-4853-832f-e912c92ea068" path="/var/lib/kubelet/pods/a98e91b2-25b6-4853-832f-e912c92ea068/volumes"
Oct 30 19:15:54.428292 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.428227    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee31e82e-3665-4a8c-b502-429976b05fd9" path="/var/lib/kubelet/pods/ee31e82e-3665-4a8c-b502-429976b05fd9/volumes"
Oct 30 19:15:54.473607 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.473557    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-maas-api-tls\") pod \"0a2211bc-9633-49ee-b65e-7c39f753357d\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") "
Oct 30 19:15:54.473860 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.473839    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a2211bc-9633-49ee-b65e-7c39f753357d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2211bc-9633-49ee-b65e-7c39f753357d-openshift-service-ca\") pod \"0a2211bc-9633-49ee-b65e-7c39f753357d\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") "
Oct 30 19:15:54.473968 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.473876    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a2211bc-9633-49ee-b65e-7c39f753357d-kube-api-access-lhf5q\" (UniqueName: \"kubernetes.io/projected/0a2211bc-9633-49ee-b65e-7c39f753357d-kube-api-access-lhf5q\") pod \"0a2211bc-9633-49ee-b65e-7c39f753357d\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") "
Oct 30 19:15:54.473968 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.473939    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-metrics-certs\") pod \"0a2211bc-9633-49ee-b65e-7c39f753357d\" (UID: \"0a2211bc-9633-49ee-b65e-7c39f753357d\") "
Oct 30 19:15:54.475180 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.475153    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a2211bc-9633-49ee-b65e-7c39f753357d-openshift-service-ca" pod "0a2211bc-9633-49ee-b65e-7c39f753357d" (UID: "0a2211bc-9633-49ee-b65e-7c39f753357d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:54.488125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.486177    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:54.494512 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.494487    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-metrics-certs" pod "0a2211bc-9633-49ee-b65e-7c39f753357d" (UID: "0a2211bc-9633-49ee-b65e-7c39f753357d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:54.494717 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.494690    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-maas-api-tls" pod "0a2211bc-9633-49ee-b65e-7c39f753357d" (UID: "0a2211bc-9633-49ee-b65e-7c39f753357d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:54.495983 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.495961    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a2211bc-9633-49ee-b65e-7c39f753357d-kube-api-access-lhf5q" pod "0a2211bc-9633-49ee-b65e-7c39f753357d" (UID: "0a2211bc-9633-49ee-b65e-7c39f753357d"). InnerVolumeSpecName "kube-api-access-lhf5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:54.576058 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.575977    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a2211bc-9633-49ee-b65e-7c39f753357d-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:54.576058 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.576010    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhf5q\" (UniqueName: \"kubernetes.io/projected/0a2211bc-9633-49ee-b65e-7c39f753357d-kube-api-access-lhf5q\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:54.576058 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.576021    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:54.576058 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.576029    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a2211bc-9633-49ee-b65e-7c39f753357d-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:54.672974 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.672943    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:54.695418 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.695389    4925 generic.go:356] "Generic (PLEG): container finished" podID="0a2211bc-9633-49ee-b65e-7c39f753357d" containerID="05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029" exitCode=0
Oct 30 19:15:54.695846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.695449    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"
Oct 30 19:15:54.695846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.695472    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr" event={"ID":"0a2211bc-9633-49ee-b65e-7c39f753357d","Type":"ContainerDied","Data":"05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029"}
Oct 30 19:15:54.695846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.695528    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr" event={"ID":"0a2211bc-9633-49ee-b65e-7c39f753357d","Type":"ContainerDied","Data":"deee0b2851747e3b51b81b48ae294fabe379149b36775232c3a45b9a4f9b8742"}
Oct 30 19:15:54.695846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.695553    4925 scope.go:122] "RemoveContainer" containerID="05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029"
Oct 30 19:15:54.706928 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.706913    4925 scope.go:122] "RemoveContainer" containerID="05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029"
Oct 30 19:15:54.707268 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:54.707243    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029\": container with ID starting with 05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029 not found: ID does not exist" containerID="05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029"
Oct 30 19:15:54.707344 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.707282    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029"} err="failed to get container status \"05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029\": rpc error: code = NotFound desc = could not find container \"05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029\": container with ID starting with 05b9bc194807c464a2bcdb4292851a85399e7dbb7658992a47fff3e0c47b2029 not found: ID does not exist"
Oct 30 19:15:54.724790 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.724767    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"]
Oct 30 19:15:54.731077 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:54.731056    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-ed012e-55c7b9c759-gqkfr"]
Oct 30 19:15:55.205301 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:55.205269    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:55.876732 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:55.876700    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:56.433138 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:56.429892    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a2211bc-9633-49ee-b65e-7c39f753357d" path="/var/lib/kubelet/pods/0a2211bc-9633-49ee-b65e-7c39f753357d/volumes"
Oct 30 19:15:56.479073 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:56.479034    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:56.657520 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:56.657487    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:57.152167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:57.147777    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:57.614436 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:57.614400    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl" podUID="3c79b524-09c4-4e29-8a1c-2306621221db" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:57.614595 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:57.614446    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv" podUID="2640e252-a00f-462c-bd40-760ba7bc79e0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:57.782059 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:57.782019    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:58.326983 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:58.326930    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:58.516117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:58.515569    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:58.699389 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:58.699353    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:58.900133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:58.889703    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:58.924126 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:58.918317    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main" containerID="cri-o://8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc" gracePeriod=45
Oct 30 19:15:58.939120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:58.927180    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main" containerID="cri-o://a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6" gracePeriod=45
Oct 30 19:15:59.069226 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:59.069142    4925 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82e8999f_b612_48bf_a4ec_c03fb7597b04.slice/crio-a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82e8999f_b612_48bf_a4ec_c03fb7597b04.slice/crio-conmon-a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3958fcc7_db32_4565_a937_8fccb3bf644e.slice/crio-8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 19:15:59.202647 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.202479    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:59.279440 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.279085    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:15:59.325350 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.325322    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 19:15:59.416709 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.416670    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-home\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-home\") pod \"3958fcc7-db32-4565-a937-8fccb3bf644e\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") "
Oct 30 19:15:59.417215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.416750    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-model-cache\") pod \"3958fcc7-db32-4565-a937-8fccb3bf644e\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") "
Oct 30 19:15:59.417215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.416813    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-tmp-dir\") pod \"82e8999f-b612-48bf-a4ec-c03fb7597b04\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") "
Oct 30 19:15:59.417215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.416849    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-dshm\") pod \"3958fcc7-db32-4565-a937-8fccb3bf644e\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") "
Oct 30 19:15:59.417215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.416918    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-dshm\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-dshm\") pod \"82e8999f-b612-48bf-a4ec-c03fb7597b04\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") "
Oct 30 19:15:59.417215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.416956    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3958fcc7-db32-4565-a937-8fccb3bf644e-tls-certs\" (UniqueName: \"kubernetes.io/secret/3958fcc7-db32-4565-a937-8fccb3bf644e-tls-certs\") pod \"3958fcc7-db32-4565-a937-8fccb3bf644e\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") "
Oct 30 19:15:59.417215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.416976    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-home" pod "3958fcc7-db32-4565-a937-8fccb3bf644e" (UID: "3958fcc7-db32-4565-a937-8fccb3bf644e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:59.417215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.416993    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-tmp-dir\") pod \"3958fcc7-db32-4565-a937-8fccb3bf644e\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") "
Oct 30 19:15:59.417215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.417041    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-home\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-home\") pod \"82e8999f-b612-48bf-a4ec-c03fb7597b04\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") "
Oct 30 19:15:59.417215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.417018    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-model-cache" pod "3958fcc7-db32-4565-a937-8fccb3bf644e" (UID: "3958fcc7-db32-4565-a937-8fccb3bf644e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:59.417215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.417076    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-tmp-dir" pod "82e8999f-b612-48bf-a4ec-c03fb7597b04" (UID: "82e8999f-b612-48bf-a4ec-c03fb7597b04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:59.417215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.417164    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/82e8999f-b612-48bf-a4ec-c03fb7597b04-tls-certs\" (UniqueName: \"kubernetes.io/secret/82e8999f-b612-48bf-a4ec-c03fb7597b04-tls-certs\") pod \"82e8999f-b612-48bf-a4ec-c03fb7597b04\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") "
Oct 30 19:15:59.417215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.417202    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-model-cache\") pod \"82e8999f-b612-48bf-a4ec-c03fb7597b04\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") "
Oct 30 19:15:59.417840 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.417247    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82e8999f-b612-48bf-a4ec-c03fb7597b04-kube-api-access-n8nkn\" (UniqueName: \"kubernetes.io/projected/82e8999f-b612-48bf-a4ec-c03fb7597b04-kube-api-access-n8nkn\") pod \"82e8999f-b612-48bf-a4ec-c03fb7597b04\" (UID: \"82e8999f-b612-48bf-a4ec-c03fb7597b04\") "
Oct 30 19:15:59.417840 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.417279    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-tmp-dir" pod "3958fcc7-db32-4565-a937-8fccb3bf644e" (UID: "3958fcc7-db32-4565-a937-8fccb3bf644e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:59.417840 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.417352    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-home" pod "82e8999f-b612-48bf-a4ec-c03fb7597b04" (UID: "82e8999f-b612-48bf-a4ec-c03fb7597b04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:59.417840 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.417438    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-model-cache" pod "82e8999f-b612-48bf-a4ec-c03fb7597b04" (UID: "82e8999f-b612-48bf-a4ec-c03fb7597b04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:59.417840 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.417453    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3958fcc7-db32-4565-a937-8fccb3bf644e-kube-api-access-xx2c6\" (UniqueName: \"kubernetes.io/projected/3958fcc7-db32-4565-a937-8fccb3bf644e-kube-api-access-xx2c6\") pod \"3958fcc7-db32-4565-a937-8fccb3bf644e\" (UID: \"3958fcc7-db32-4565-a937-8fccb3bf644e\") "
Oct 30 19:15:59.418410 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.418388    4925 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-home\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.418500 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.418416    4925 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-model-cache\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.418500 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.418435    4925 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-tmp-dir\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.418500 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.418449    4925 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-tmp-dir\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.418500 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.418463    4925 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-home\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.418500 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.418476    4925 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-model-cache\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.420066 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.420035    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82e8999f-b612-48bf-a4ec-c03fb7597b04-tls-certs" pod "82e8999f-b612-48bf-a4ec-c03fb7597b04" (UID: "82e8999f-b612-48bf-a4ec-c03fb7597b04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:59.420218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.420090    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-dshm" pod "3958fcc7-db32-4565-a937-8fccb3bf644e" (UID: "3958fcc7-db32-4565-a937-8fccb3bf644e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:59.420218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.420150    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3958fcc7-db32-4565-a937-8fccb3bf644e-tls-certs" pod "3958fcc7-db32-4565-a937-8fccb3bf644e" (UID: "3958fcc7-db32-4565-a937-8fccb3bf644e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:59.420218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.420162    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82e8999f-b612-48bf-a4ec-c03fb7597b04-kube-api-access-n8nkn" pod "82e8999f-b612-48bf-a4ec-c03fb7597b04" (UID: "82e8999f-b612-48bf-a4ec-c03fb7597b04"). InnerVolumeSpecName "kube-api-access-n8nkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:59.420371 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.420338    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3958fcc7-db32-4565-a937-8fccb3bf644e-kube-api-access-xx2c6" pod "3958fcc7-db32-4565-a937-8fccb3bf644e" (UID: "3958fcc7-db32-4565-a937-8fccb3bf644e"). InnerVolumeSpecName "kube-api-access-xx2c6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:59.420577 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.420557    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-dshm" pod "82e8999f-b612-48bf-a4ec-c03fb7597b04" (UID: "82e8999f-b612-48bf-a4ec-c03fb7597b04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:59.520485 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.520400    4925 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3958fcc7-db32-4565-a937-8fccb3bf644e-dshm\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.520485 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.520434    4925 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82e8999f-b612-48bf-a4ec-c03fb7597b04-dshm\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.520485 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.520450    4925 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3958fcc7-db32-4565-a937-8fccb3bf644e-tls-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.520485 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.520460    4925 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82e8999f-b612-48bf-a4ec-c03fb7597b04-tls-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.520485 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.520469    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8nkn\" (UniqueName: \"kubernetes.io/projected/82e8999f-b612-48bf-a4ec-c03fb7597b04-kube-api-access-n8nkn\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.520485 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.520480    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx2c6\" (UniqueName: \"kubernetes.io/projected/3958fcc7-db32-4565-a937-8fccb3bf644e-kube-api-access-xx2c6\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:59.780630 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.780535    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:59.781413 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.781387    4925 generic.go:356] "Generic (PLEG): container finished" podID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerID="a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6" exitCode=0
Oct 30 19:15:59.794155 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.781721    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" event={"ID":"82e8999f-b612-48bf-a4ec-c03fb7597b04","Type":"ContainerDied","Data":"a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6"}
Oct 30 19:15:59.794972 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.794939    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8" event={"ID":"82e8999f-b612-48bf-a4ec-c03fb7597b04","Type":"ContainerDied","Data":"1d3039a3eeccd0ce8b06bef3989569996a41eb983dddcafc83c7e917dd6afa76"}
Oct 30 19:15:59.795156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.795141    4925 scope.go:122] "RemoveContainer" containerID="a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6"
Oct 30 19:15:59.795349 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.791239    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"
Oct 30 19:15:59.805894 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.805862    4925 generic.go:356] "Generic (PLEG): container finished" podID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerID="8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc" exitCode=0
Oct 30 19:15:59.806080 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.806060    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"
Oct 30 19:15:59.811454 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.811426    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" event={"ID":"3958fcc7-db32-4565-a937-8fccb3bf644e","Type":"ContainerDied","Data":"8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc"}
Oct 30 19:15:59.811566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.811473    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj" event={"ID":"3958fcc7-db32-4565-a937-8fccb3bf644e","Type":"ContainerDied","Data":"b7fa7dd1a137df582f66f24330779f33d762573f164ec86aa0ff09c728ce62c8"}
Oct 30 19:15:59.823834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.823811    4925 scope.go:122] "RemoveContainer" containerID="a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6"
Oct 30 19:15:59.824242 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:59.824223    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6\": container with ID starting with a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6 not found: ID does not exist" containerID="a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6"
Oct 30 19:15:59.824294 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.824252    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6"} err="failed to get container status \"a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6\": rpc error: code = NotFound desc = could not find container \"a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6\": container with ID starting with a602da0fbd3728c9f631db25eb02000fe9e6aa5b9c5e1e81a8a1e0b5320ab7b6 not found: ID does not exist"
Oct 30 19:15:59.824294 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.824269    4925 scope.go:122] "RemoveContainer" containerID="8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc"
Oct 30 19:15:59.831182 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.830933    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"]
Oct 30 19:15:59.835215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.835192    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-9de872/distinct-2-w0-9de872-kserve-5d87dc9767-nm7q8"]
Oct 30 19:15:59.837633 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.837612    4925 scope.go:122] "RemoveContainer" containerID="8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc"
Oct 30 19:15:59.837986 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:15:59.837961    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc\": container with ID starting with 8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc not found: ID does not exist" containerID="8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc"
Oct 30 19:15:59.838070 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.837989    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc"} err="failed to get container status \"8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc\": rpc error: code = NotFound desc = could not find container \"8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc\": container with ID starting with 8e14227e5f720b79cff9701b502ceda288b16c7fcb36eb5cf375c4a3706675dc not found: ID does not exist"
Oct 30 19:15:59.849172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.849150    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"]
Oct 30 19:15:59.850843 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:15:59.850825    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-9de872/premium-w0-9de872-kserve-575494f949-5sqdj"]
Oct 30 19:16:00.138665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:00.138575    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:00.340233 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:00.340189    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:00.425456 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:00.425424    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" path="/var/lib/kubelet/pods/3958fcc7-db32-4565-a937-8fccb3bf644e/volumes"
Oct 30 19:16:00.426179 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:00.426157    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" path="/var/lib/kubelet/pods/82e8999f-b612-48bf-a4ec-c03fb7597b04/volumes"
Oct 30 19:16:00.560053 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:00.560017    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:00.867290 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:00.867019    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:01.285506 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:01.275044    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg" podUID="af4ba1e6-4f4c-42c6-bc34-4eafcc18e428" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:01.291191 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:01.288838    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh" podUID="3107703b-0cfb-4320-84c9-d023eeea6c36" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:01.953196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:01.949185    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:02.268121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:02.266909    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:02.458377 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:02.458023    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:02.459284 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:02.459259    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"]
Oct 30 19:16:02.459495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:02.459471    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" podUID="cfe33049-c39b-45f7-97c2-78dbd003c401" containerName="istio-proxy" containerID="cri-o://fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18" gracePeriod=30
Oct 30 19:16:02.642380 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:02.642275    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:03.000812 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:03.000771    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:03.184050 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:03.184011    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:04.157768 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:04.157734    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:04.781119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:04.780496    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:05.435073 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:05.435036    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:05.586694 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:05.586661    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"]
Oct 30 19:16:05.648760 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:05.648730    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:05.891780 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:05.891692    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:06.172481 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:06.172447    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:06.342171 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:06.342130    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"]
Oct 30 19:16:06.342421 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:06.342392    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" podUID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" containerName="istio-proxy" containerID="cri-o://cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4" gracePeriod=30
Oct 30 19:16:06.363647 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:06.363620    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:06.548953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:06.548829    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:07.334338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.328314    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:07.613839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.613753    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl" podUID="3c79b524-09c4-4e29-8a1c-2306621221db" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:07.614316 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.614252    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv" podUID="2640e252-a00f-462c-bd40-760ba7bc79e0" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:07.759432 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.759406    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:16:07.906463 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.906372    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"]
Oct 30 19:16:07.919828 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.919795    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-envoy\") pod \"cfe33049-c39b-45f7-97c2-78dbd003c401\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") "
Oct 30 19:16:07.919997 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.919979    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-socket\") pod \"cfe33049-c39b-45f7-97c2-78dbd003c401\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") "
Oct 30 19:16:07.920078 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.920019    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-podinfo\") pod \"cfe33049-c39b-45f7-97c2-78dbd003c401\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") "
Oct 30 19:16:07.920078 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.920065    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-data\") pod \"cfe33049-c39b-45f7-97c2-78dbd003c401\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") "
Oct 30 19:16:07.920273 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.920239    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-certs\") pod \"cfe33049-c39b-45f7-97c2-78dbd003c401\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") "
Oct 30 19:16:07.920334 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.920288    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-kube-api-access-pggxk\" (UniqueName: \"kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-kube-api-access-pggxk\") pod \"cfe33049-c39b-45f7-97c2-78dbd003c401\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") "
Oct 30 19:16:07.920417 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.920397    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-credential-socket\") pod \"cfe33049-c39b-45f7-97c2-78dbd003c401\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") "
Oct 30 19:16:07.920489 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.920436    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cfe33049-c39b-45f7-97c2-78dbd003c401-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfe33049-c39b-45f7-97c2-78dbd003c401-istiod-ca-cert\") pod \"cfe33049-c39b-45f7-97c2-78dbd003c401\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") "
Oct 30 19:16:07.920544 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.920490    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-token\" (UniqueName: \"kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-token\") pod \"cfe33049-c39b-45f7-97c2-78dbd003c401\" (UID: \"cfe33049-c39b-45f7-97c2-78dbd003c401\") "
Oct 30 19:16:07.920603 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.920555    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-socket" pod "cfe33049-c39b-45f7-97c2-78dbd003c401" (UID: "cfe33049-c39b-45f7-97c2-78dbd003c401"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:07.921554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.920889    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-data" pod "cfe33049-c39b-45f7-97c2-78dbd003c401" (UID: "cfe33049-c39b-45f7-97c2-78dbd003c401"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:07.921554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.921173    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:07.921554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.921195    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:07.922855 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.922700    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-certs" pod "cfe33049-c39b-45f7-97c2-78dbd003c401" (UID: "cfe33049-c39b-45f7-97c2-78dbd003c401"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:07.922855 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.922828    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfe33049-c39b-45f7-97c2-78dbd003c401-istiod-ca-cert" pod "cfe33049-c39b-45f7-97c2-78dbd003c401" (UID: "cfe33049-c39b-45f7-97c2-78dbd003c401"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:07.923359 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.923326    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-credential-socket" pod "cfe33049-c39b-45f7-97c2-78dbd003c401" (UID: "cfe33049-c39b-45f7-97c2-78dbd003c401"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:07.924000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.923884    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-token" pod "cfe33049-c39b-45f7-97c2-78dbd003c401" (UID: "cfe33049-c39b-45f7-97c2-78dbd003c401"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:07.927678 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.927643    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-kube-api-access-pggxk" pod "cfe33049-c39b-45f7-97c2-78dbd003c401" (UID: "cfe33049-c39b-45f7-97c2-78dbd003c401"). InnerVolumeSpecName "kube-api-access-pggxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:07.938513 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.938465    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-envoy" pod "cfe33049-c39b-45f7-97c2-78dbd003c401" (UID: "cfe33049-c39b-45f7-97c2-78dbd003c401"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:07.938604 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.938566    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-podinfo" pod "cfe33049-c39b-45f7-97c2-78dbd003c401" (UID: "cfe33049-c39b-45f7-97c2-78dbd003c401"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:16:07.960352 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.960327    4925 generic.go:356] "Generic (PLEG): container finished" podID="cfe33049-c39b-45f7-97c2-78dbd003c401" containerID="fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18" exitCode=0
Oct 30 19:16:07.960490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.960391    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"
Oct 30 19:16:07.960490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.960421    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" event={"ID":"cfe33049-c39b-45f7-97c2-78dbd003c401","Type":"ContainerDied","Data":"fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18"}
Oct 30 19:16:07.960490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.960471    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf" event={"ID":"cfe33049-c39b-45f7-97c2-78dbd003c401","Type":"ContainerDied","Data":"fdd7978fb94732d9a04b9438883c3abb47e44cf037cfed32561f33ab7e6f3ecb"}
Oct 30 19:16:07.960672 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.960499    4925 scope.go:122] "RemoveContainer" containerID="fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18"
Oct 30 19:16:07.972670 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.972643    4925 scope.go:122] "RemoveContainer" containerID="fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18"
Oct 30 19:16:07.972966 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:07.972948    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18\": container with ID starting with fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18 not found: ID does not exist" containerID="fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18"
Oct 30 19:16:07.973058 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.972976    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18"} err="failed to get container status \"fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18\": rpc error: code = NotFound desc = could not find container \"fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18\": container with ID starting with fa8640931570330bb69795d2b7af2db598602fb19c0c822aaf17e9b444298f18 not found: ID does not exist"
Oct 30 19:16:07.997544 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:07.997518    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"]
Oct 30 19:16:08.002992 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.002969    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-ed012e-openshift-default-6ddfc957bc-zldwf"]
Oct 30 19:16:08.022209 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.022188    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.022318 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.022210    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pggxk\" (UniqueName: \"kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-kube-api-access-pggxk\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.022318 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.022225    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.022318 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.022234    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfe33049-c39b-45f7-97c2-78dbd003c401-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.022318 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.022244    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.022318 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.022252    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.022318 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.022260    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfe33049-c39b-45f7-97c2-78dbd003c401-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.116047 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.116008    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:08.321160 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.321125    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:08.427317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.427288    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfe33049-c39b-45f7-97c2-78dbd003c401" path="/var/lib/kubelet/pods/cfe33049-c39b-45f7-97c2-78dbd003c401/volumes"
Oct 30 19:16:08.444850 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.444824    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"
Oct 30 19:16:08.449919 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.449895    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"
Oct 30 19:16:08.544125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.538728    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3107703b-0cfb-4320-84c9-d023eeea6c36-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3107703b-0cfb-4320-84c9-d023eeea6c36-plugins-config-volume\") pod \"3107703b-0cfb-4320-84c9-d023eeea6c36\" (UID: \"3107703b-0cfb-4320-84c9-d023eeea6c36\") "
Oct 30 19:16:08.544125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.538864    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-kube-api-access-xphpz\" (UniqueName: \"kubernetes.io/projected/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-kube-api-access-xphpz\") pod \"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428\" (UID: \"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428\") "
Oct 30 19:16:08.544125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.539005    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-plugins-config-volume\") pod \"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428\" (UID: \"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428\") "
Oct 30 19:16:08.544125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.539043    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3107703b-0cfb-4320-84c9-d023eeea6c36-kube-api-access-rjpzr\" (UniqueName: \"kubernetes.io/projected/3107703b-0cfb-4320-84c9-d023eeea6c36-kube-api-access-rjpzr\") pod \"3107703b-0cfb-4320-84c9-d023eeea6c36\" (UID: \"3107703b-0cfb-4320-84c9-d023eeea6c36\") "
Oct 30 19:16:08.544125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.540495    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3107703b-0cfb-4320-84c9-d023eeea6c36-plugins-config-volume" pod "3107703b-0cfb-4320-84c9-d023eeea6c36" (UID: "3107703b-0cfb-4320-84c9-d023eeea6c36"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:08.544125 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.541587    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-plugins-config-volume" pod "af4ba1e6-4f4c-42c6-bc34-4eafcc18e428" (UID: "af4ba1e6-4f4c-42c6-bc34-4eafcc18e428"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:08.545788 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.545758    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3107703b-0cfb-4320-84c9-d023eeea6c36-kube-api-access-rjpzr" pod "3107703b-0cfb-4320-84c9-d023eeea6c36" (UID: "3107703b-0cfb-4320-84c9-d023eeea6c36"). InnerVolumeSpecName "kube-api-access-rjpzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:08.551714 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.551689    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-kube-api-access-xphpz" pod "af4ba1e6-4f4c-42c6-bc34-4eafcc18e428" (UID: "af4ba1e6-4f4c-42c6-bc34-4eafcc18e428"). InnerVolumeSpecName "kube-api-access-xphpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:08.643498 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.642349    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3107703b-0cfb-4320-84c9-d023eeea6c36-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.643498 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.642389    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xphpz\" (UniqueName: \"kubernetes.io/projected/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-kube-api-access-xphpz\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.643498 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.642406    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.643498 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.642420    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjpzr\" (UniqueName: \"kubernetes.io/projected/3107703b-0cfb-4320-84c9-d023eeea6c36-kube-api-access-rjpzr\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.699984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.699961    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"
Oct 30 19:16:08.704859 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.704828    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"
Oct 30 19:16:08.845401 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.845369    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c79b524-09c4-4e29-8a1c-2306621221db-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c79b524-09c4-4e29-8a1c-2306621221db-plugins-config-volume\") pod \"3c79b524-09c4-4e29-8a1c-2306621221db\" (UID: \"3c79b524-09c4-4e29-8a1c-2306621221db\") "
Oct 30 19:16:08.845590 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.845439    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2640e252-a00f-462c-bd40-760ba7bc79e0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2640e252-a00f-462c-bd40-760ba7bc79e0-plugins-config-volume\") pod \"2640e252-a00f-462c-bd40-760ba7bc79e0\" (UID: \"2640e252-a00f-462c-bd40-760ba7bc79e0\") "
Oct 30 19:16:08.845590 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.845509    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c79b524-09c4-4e29-8a1c-2306621221db-kube-api-access-7brkp\" (UniqueName: \"kubernetes.io/projected/3c79b524-09c4-4e29-8a1c-2306621221db-kube-api-access-7brkp\") pod \"3c79b524-09c4-4e29-8a1c-2306621221db\" (UID: \"3c79b524-09c4-4e29-8a1c-2306621221db\") "
Oct 30 19:16:08.845703 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.845618    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2640e252-a00f-462c-bd40-760ba7bc79e0-kube-api-access-2smp9\" (UniqueName: \"kubernetes.io/projected/2640e252-a00f-462c-bd40-760ba7bc79e0-kube-api-access-2smp9\") pod \"2640e252-a00f-462c-bd40-760ba7bc79e0\" (UID: \"2640e252-a00f-462c-bd40-760ba7bc79e0\") "
Oct 30 19:16:08.845804 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.845772    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2640e252-a00f-462c-bd40-760ba7bc79e0-plugins-config-volume" pod "2640e252-a00f-462c-bd40-760ba7bc79e0" (UID: "2640e252-a00f-462c-bd40-760ba7bc79e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:08.845804 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.845797    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c79b524-09c4-4e29-8a1c-2306621221db-plugins-config-volume" pod "3c79b524-09c4-4e29-8a1c-2306621221db" (UID: "3c79b524-09c4-4e29-8a1c-2306621221db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:08.846284 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.846111    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c79b524-09c4-4e29-8a1c-2306621221db-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.846284 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.846134    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2640e252-a00f-462c-bd40-760ba7bc79e0-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.847792 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.847770    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c79b524-09c4-4e29-8a1c-2306621221db-kube-api-access-7brkp" pod "3c79b524-09c4-4e29-8a1c-2306621221db" (UID: "3c79b524-09c4-4e29-8a1c-2306621221db"). InnerVolumeSpecName "kube-api-access-7brkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:08.847881 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.847791    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2640e252-a00f-462c-bd40-760ba7bc79e0-kube-api-access-2smp9" pod "2640e252-a00f-462c-bd40-760ba7bc79e0" (UID: "2640e252-a00f-462c-bd40-760ba7bc79e0"). InnerVolumeSpecName "kube-api-access-2smp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:08.958205 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.955341    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7brkp\" (UniqueName: \"kubernetes.io/projected/3c79b524-09c4-4e29-8a1c-2306621221db-kube-api-access-7brkp\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.958205 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.955385    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2smp9\" (UniqueName: \"kubernetes.io/projected/2640e252-a00f-462c-bd40-760ba7bc79e0-kube-api-access-2smp9\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:08.982664 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.982371    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:08.986397 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.986367    4925 generic.go:356] "Generic (PLEG): container finished" podID="af4ba1e6-4f4c-42c6-bc34-4eafcc18e428" containerID="b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d" exitCode=0
Oct 30 19:16:08.986551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.986415    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"
Oct 30 19:16:08.986551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.986456    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg" event={"ID":"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428","Type":"ContainerDied","Data":"b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d"}
Oct 30 19:16:08.986551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.986508    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg" event={"ID":"af4ba1e6-4f4c-42c6-bc34-4eafcc18e428","Type":"ContainerDied","Data":"234052df403dde5dc5b4a64085279f9d434424b4f1d3b4dc39b73f5cca25e678"}
Oct 30 19:16:08.986551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.986529    4925 scope.go:122] "RemoveContainer" containerID="b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d"
Oct 30 19:16:08.989688 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.989652    4925 generic.go:356] "Generic (PLEG): container finished" podID="3107703b-0cfb-4320-84c9-d023eeea6c36" containerID="ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3" exitCode=0
Oct 30 19:16:08.989815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.989699    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"
Oct 30 19:16:08.989815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.989759    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh" event={"ID":"3107703b-0cfb-4320-84c9-d023eeea6c36","Type":"ContainerDied","Data":"ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3"}
Oct 30 19:16:08.989815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.989797    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh" event={"ID":"3107703b-0cfb-4320-84c9-d023eeea6c36","Type":"ContainerDied","Data":"cad87d1fb29418700054e3ca730bcac7e2a05d16d9a7f12cb73b7c46c51ed6bc"}
Oct 30 19:16:08.995189 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.995166    4925 generic.go:356] "Generic (PLEG): container finished" podID="3c79b524-09c4-4e29-8a1c-2306621221db" containerID="2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af" exitCode=0
Oct 30 19:16:08.995311 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.995213    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"
Oct 30 19:16:08.995311 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.995252    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl" event={"ID":"3c79b524-09c4-4e29-8a1c-2306621221db","Type":"ContainerDied","Data":"2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af"}
Oct 30 19:16:08.995311 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:08.995301    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl" event={"ID":"3c79b524-09c4-4e29-8a1c-2306621221db","Type":"ContainerDied","Data":"50afc861bcc2fcb40eba225b921a58d9893f4e91dd441ec52200db56e88bb91f"}
Oct 30 19:16:09.000458 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.000434    4925 scope.go:122] "RemoveContainer" containerID="b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d"
Oct 30 19:16:09.000724 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:09.000706    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d\": container with ID starting with b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d not found: ID does not exist" containerID="b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d"
Oct 30 19:16:09.000855 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.000830    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d"} err="failed to get container status \"b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d\": rpc error: code = NotFound desc = could not find container \"b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d\": container with ID starting with b5cff43b854fb689c4b9d2630b014403aded3f1e2c905dbe4762af45567ba30d not found: ID does not exist"
Oct 30 19:16:09.001226 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.001210    4925 scope.go:122] "RemoveContainer" containerID="ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3"
Oct 30 19:16:09.001761 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.001737    4925 generic.go:356] "Generic (PLEG): container finished" podID="2640e252-a00f-462c-bd40-760ba7bc79e0" containerID="52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d" exitCode=0
Oct 30 19:16:09.001866 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.001844    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv" event={"ID":"2640e252-a00f-462c-bd40-760ba7bc79e0","Type":"ContainerDied","Data":"52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d"}
Oct 30 19:16:09.001936 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.001883    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv" event={"ID":"2640e252-a00f-462c-bd40-760ba7bc79e0","Type":"ContainerDied","Data":"c7b845d17d867b3d8739600a3e551aa05768b6277566cff6c8af5b290cb45445"}
Oct 30 19:16:09.002001 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.001946    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"
Oct 30 19:16:09.015337 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.015314    4925 scope.go:122] "RemoveContainer" containerID="ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3"
Oct 30 19:16:09.015648 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:09.015628    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3\": container with ID starting with ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3 not found: ID does not exist" containerID="ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3"
Oct 30 19:16:09.015714 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.015655    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3"} err="failed to get container status \"ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3\": rpc error: code = NotFound desc = could not find container \"ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3\": container with ID starting with ea218dbe6b1dc4dc124668a42d3f8b0822a893ef4b00df5033c7606c3d8f87c3 not found: ID does not exist"
Oct 30 19:16:09.015714 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.015672    4925 scope.go:122] "RemoveContainer" containerID="2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af"
Oct 30 19:16:09.017792 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.017771    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"]
Oct 30 19:16:09.023241 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.023217    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-c2e3f9-5797bffc67-tmhgg"]
Oct 30 19:16:09.026584 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.026559    4925 scope.go:122] "RemoveContainer" containerID="2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af"
Oct 30 19:16:09.026903 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:09.026881    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af\": container with ID starting with 2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af not found: ID does not exist" containerID="2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af"
Oct 30 19:16:09.026982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.026907    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af"} err="failed to get container status \"2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af\": rpc error: code = NotFound desc = could not find container \"2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af\": container with ID starting with 2780e2865f902bcebff6438fa6405cf2cd96c770b1c44591fd5112d0d24dc6af not found: ID does not exist"
Oct 30 19:16:09.026982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.026925    4925 scope.go:122] "RemoveContainer" containerID="52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d"
Oct 30 19:16:09.035892 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.035873    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"]
Oct 30 19:16:09.040137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.040041    4925 scope.go:122] "RemoveContainer" containerID="52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d"
Oct 30 19:16:09.041075 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.041051    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-c2e3f9-5889544686-l6xbh"]
Oct 30 19:16:09.042951 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:09.042925    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d\": container with ID starting with 52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d not found: ID does not exist" containerID="52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d"
Oct 30 19:16:09.043064 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.042960    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d"} err="failed to get container status \"52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d\": rpc error: code = NotFound desc = could not find container \"52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d\": container with ID starting with 52a1aa353e190563ac08c5c9d9ab0213122a454071afc27a4a7c5374fdd3046d not found: ID does not exist"
Oct 30 19:16:09.056568 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.056548    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"]
Oct 30 19:16:09.063677 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.063660    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w4-ed012e-6cc8bdf976-z2pfl"]
Oct 30 19:16:09.074338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.074302    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"]
Oct 30 19:16:09.083389 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.083362    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-ed012e-775dc4b9bc-ztnzv"]
Oct 30 19:16:09.459887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.459839    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"]
Oct 30 19:16:09.460140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.460111    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87" podUID="bfd0603c-3ee0-476c-916c-e9c4f49b1077" containerName="maas-api" containerID="cri-o://c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d" gracePeriod=30
Oct 30 19:16:09.604229 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.604194    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:09.913728 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.866402    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:09.990047 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:09.990020    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 19:16:10.019087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.019040    4925 generic.go:356] "Generic (PLEG): container finished" podID="bfd0603c-3ee0-476c-916c-e9c4f49b1077" containerID="c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d" exitCode=0
Oct 30 19:16:10.019294 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.019184    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87" event={"ID":"bfd0603c-3ee0-476c-916c-e9c4f49b1077","Type":"ContainerDied","Data":"c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d"}
Oct 30 19:16:10.019294 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.019242    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87" event={"ID":"bfd0603c-3ee0-476c-916c-e9c4f49b1077","Type":"ContainerDied","Data":"8d5faf561f0f4faca34a5e529198a12263059a8dfe5fd62e09704ae3982f41cf"}
Oct 30 19:16:10.019294 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.019272    4925 scope.go:122] "RemoveContainer" containerID="c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d"
Oct 30 19:16:10.019476 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.019403    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"
Oct 30 19:16:10.047286 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.047254    4925 scope.go:122] "RemoveContainer" containerID="c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d"
Oct 30 19:16:10.047739 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:10.047694    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d\": container with ID starting with c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d not found: ID does not exist" containerID="c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d"
Oct 30 19:16:10.047865 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.047738    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d"} err="failed to get container status \"c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d\": rpc error: code = NotFound desc = could not find container \"c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d\": container with ID starting with c479430f0ddddf456a9c0aa9be114b6178cee6233c09dfecdf5a54bc6bfe677d not found: ID does not exist"
Oct 30 19:16:10.110053 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.110023    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bfd0603c-3ee0-476c-916c-e9c4f49b1077-kube-api-access-gjwwd\" (UniqueName: \"kubernetes.io/projected/bfd0603c-3ee0-476c-916c-e9c4f49b1077-kube-api-access-gjwwd\") pod \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") "
Oct 30 19:16:10.110267 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.110246    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-maas-api-tls\") pod \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") "
Oct 30 19:16:10.110342 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.110327    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-metrics-certs\") pod \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") "
Oct 30 19:16:10.110470 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.110455    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bfd0603c-3ee0-476c-916c-e9c4f49b1077-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd0603c-3ee0-476c-916c-e9c4f49b1077-openshift-service-ca\") pod \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\" (UID: \"bfd0603c-3ee0-476c-916c-e9c4f49b1077\") "
Oct 30 19:16:10.111056 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.111023    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfd0603c-3ee0-476c-916c-e9c4f49b1077-openshift-service-ca" pod "bfd0603c-3ee0-476c-916c-e9c4f49b1077" (UID: "bfd0603c-3ee0-476c-916c-e9c4f49b1077"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:10.112390 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.112365    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfd0603c-3ee0-476c-916c-e9c4f49b1077-kube-api-access-gjwwd" pod "bfd0603c-3ee0-476c-916c-e9c4f49b1077" (UID: "bfd0603c-3ee0-476c-916c-e9c4f49b1077"). InnerVolumeSpecName "kube-api-access-gjwwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:10.112710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.112686    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-metrics-certs" pod "bfd0603c-3ee0-476c-916c-e9c4f49b1077" (UID: "bfd0603c-3ee0-476c-916c-e9c4f49b1077"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:10.112887 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.112865    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-maas-api-tls" pod "bfd0603c-3ee0-476c-916c-e9c4f49b1077" (UID: "bfd0603c-3ee0-476c-916c-e9c4f49b1077"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:10.124472 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.124445    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:10.217079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.216988    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:10.217079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.217023    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfd0603c-3ee0-476c-916c-e9c4f49b1077-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:10.217079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.217037    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfd0603c-3ee0-476c-916c-e9c4f49b1077-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:10.217079 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.217050    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjwwd\" (UniqueName: \"kubernetes.io/projected/bfd0603c-3ee0-476c-916c-e9c4f49b1077-kube-api-access-gjwwd\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:10.346684 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.346653    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:10.353631 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.353605    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"]
Oct 30 19:16:10.362384 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.362346    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87"]
Oct 30 19:16:10.423345 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.423318    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2640e252-a00f-462c-bd40-760ba7bc79e0" path="/var/lib/kubelet/pods/2640e252-a00f-462c-bd40-760ba7bc79e0/volumes"
Oct 30 19:16:10.423760 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.423744    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3107703b-0cfb-4320-84c9-d023eeea6c36" path="/var/lib/kubelet/pods/3107703b-0cfb-4320-84c9-d023eeea6c36/volumes"
Oct 30 19:16:10.424144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.424130    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c79b524-09c4-4e29-8a1c-2306621221db" path="/var/lib/kubelet/pods/3c79b524-09c4-4e29-8a1c-2306621221db/volumes"
Oct 30 19:16:10.424496 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.424484    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af4ba1e6-4f4c-42c6-bc34-4eafcc18e428" path="/var/lib/kubelet/pods/af4ba1e6-4f4c-42c6-bc34-4eafcc18e428/volumes"
Oct 30 19:16:10.425221 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.425207    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bfd0603c-3ee0-476c-916c-e9c4f49b1077" path="/var/lib/kubelet/pods/bfd0603c-3ee0-476c-916c-e9c4f49b1077/volumes"
Oct 30 19:16:10.535423 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.535342    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:10.587231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.587197    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l" podUID="82c8263f-5c5f-4cce-adcb-8e15e22a8158" containerName="payload-processing" containerID="cri-o://18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e" gracePeriod=25
Oct 30 19:16:10.888296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.888268    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:16:10.925183 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.925151    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-routing-overlay-volume\") pod \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") "
Oct 30 19:16:10.925650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.925339    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-provider-credentials\" (UniqueName: \"kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-provider-credentials\") pod \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") "
Oct 30 19:16:10.925650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.925414    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-plugins-config-volume\") pod \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") "
Oct 30 19:16:10.925650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.925508    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-kube-api-access-2v75z\" (UniqueName: \"kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-kube-api-access-2v75z\") pod \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\" (UID: \"82c8263f-5c5f-4cce-adcb-8e15e22a8158\") "
Oct 30 19:16:10.925650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.925563    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-routing-overlay-volume" pod "82c8263f-5c5f-4cce-adcb-8e15e22a8158" (UID: "82c8263f-5c5f-4cce-adcb-8e15e22a8158"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:10.925920 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.925891    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-plugins-config-volume" pod "82c8263f-5c5f-4cce-adcb-8e15e22a8158" (UID: "82c8263f-5c5f-4cce-adcb-8e15e22a8158"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:10.926109 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.926072    4925 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-routing-overlay-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:10.926213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.926118    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82c8263f-5c5f-4cce-adcb-8e15e22a8158-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:10.929148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.928288    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-kube-api-access-2v75z" pod "82c8263f-5c5f-4cce-adcb-8e15e22a8158" (UID: "82c8263f-5c5f-4cce-adcb-8e15e22a8158"). InnerVolumeSpecName "kube-api-access-2v75z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:10.929148 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:10.928311    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-provider-credentials" pod "82c8263f-5c5f-4cce-adcb-8e15e22a8158" (UID: "82c8263f-5c5f-4cce-adcb-8e15e22a8158"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:11.026957 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.026925    4925 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-provider-credentials\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.026957 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.026952    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v75z\" (UniqueName: \"kubernetes.io/projected/82c8263f-5c5f-4cce-adcb-8e15e22a8158-kube-api-access-2v75z\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.032234 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.032209    4925 generic.go:356] "Generic (PLEG): container finished" podID="82c8263f-5c5f-4cce-adcb-8e15e22a8158" containerID="18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e" exitCode=0
Oct 30 19:16:11.032364 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.032250    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"
Oct 30 19:16:11.032364 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.032273    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l" event={"ID":"82c8263f-5c5f-4cce-adcb-8e15e22a8158","Type":"ContainerDied","Data":"18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e"}
Oct 30 19:16:11.032364 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.032331    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l" event={"ID":"82c8263f-5c5f-4cce-adcb-8e15e22a8158","Type":"ContainerDied","Data":"78a0fdb023ce5f39d582701d4e6a1e9e616c53fd2f16b1ded73ead96aef86cf4"}
Oct 30 19:16:11.032364 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.032356    4925 scope.go:122] "RemoveContainer" containerID="18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e"
Oct 30 19:16:11.085948 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.085919    4925 scope.go:122] "RemoveContainer" containerID="18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e"
Oct 30 19:16:11.086634 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:11.086607    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e\": container with ID starting with 18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e not found: ID does not exist" containerID="18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e"
Oct 30 19:16:11.086777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.086644    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e"} err="failed to get container status \"18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e\": rpc error: code = NotFound desc = could not find container \"18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e\": container with ID starting with 18aa7d35a736887c4a2767a82e414679bb05639132904448843271a03e4e990e not found: ID does not exist"
Oct 30 19:16:11.118877 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.118795    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"]
Oct 30 19:16:11.128713 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.128685    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-ca2cd5/payload-processing-external-model-e2e-extmod-a-ca2cd5-85c5hsp9l"]
Oct 30 19:16:11.185158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.174801    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"]
Oct 30 19:16:11.206068 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.206045    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:11.420827 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.420796    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:11.831912 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.831880    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:16:11.855244 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.855216    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-certs\") pod \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") "
Oct 30 19:16:11.855373 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.855316    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-token\" (UniqueName: \"kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-token\") pod \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") "
Oct 30 19:16:11.855373 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.855354    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-data\") pod \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") "
Oct 30 19:16:11.855478 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.855426    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-podinfo\") pod \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") "
Oct 30 19:16:11.855544 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.855494    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-kube-api-access-vnljv\" (UniqueName: \"kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-kube-api-access-vnljv\") pod \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") "
Oct 30 19:16:11.855544 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.855527    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-certs" pod "532cf3a3-18af-48ac-8df2-ce52f16f98ea" (UID: "532cf3a3-18af-48ac-8df2-ce52f16f98ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:11.855650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.855548    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-envoy\") pod \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") "
Oct 30 19:16:11.855650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.855595    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-socket\") pod \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") "
Oct 30 19:16:11.855760 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.855718    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istiod-ca-cert\") pod \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") "
Oct 30 19:16:11.855818 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.855761    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-credential-socket\") pod \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\" (UID: \"532cf3a3-18af-48ac-8df2-ce52f16f98ea\") "
Oct 30 19:16:11.856542 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.856340    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.857588 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.856941    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-data" pod "532cf3a3-18af-48ac-8df2-ce52f16f98ea" (UID: "532cf3a3-18af-48ac-8df2-ce52f16f98ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:11.857588 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.857544    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istiod-ca-cert" pod "532cf3a3-18af-48ac-8df2-ce52f16f98ea" (UID: "532cf3a3-18af-48ac-8df2-ce52f16f98ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:11.858641 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.857920    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-socket" pod "532cf3a3-18af-48ac-8df2-ce52f16f98ea" (UID: "532cf3a3-18af-48ac-8df2-ce52f16f98ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:11.858896 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.858863    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-credential-socket" pod "532cf3a3-18af-48ac-8df2-ce52f16f98ea" (UID: "532cf3a3-18af-48ac-8df2-ce52f16f98ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:11.872978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.872025    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-envoy" pod "532cf3a3-18af-48ac-8df2-ce52f16f98ea" (UID: "532cf3a3-18af-48ac-8df2-ce52f16f98ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:11.872978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.872470    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-kube-api-access-vnljv" pod "532cf3a3-18af-48ac-8df2-ce52f16f98ea" (UID: "532cf3a3-18af-48ac-8df2-ce52f16f98ea"). InnerVolumeSpecName "kube-api-access-vnljv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:11.875065 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.875025    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-podinfo" pod "532cf3a3-18af-48ac-8df2-ce52f16f98ea" (UID: "532cf3a3-18af-48ac-8df2-ce52f16f98ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:16:11.875627 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.875601    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-token" pod "532cf3a3-18af-48ac-8df2-ce52f16f98ea" (UID: "532cf3a3-18af-48ac-8df2-ce52f16f98ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:11.957540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.957482    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.957540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.957507    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.957540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.957518    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.957540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.957530    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnljv\" (UniqueName: \"kubernetes.io/projected/532cf3a3-18af-48ac-8df2-ce52f16f98ea-kube-api-access-vnljv\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.957540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.957541    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.958000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.957551    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.958000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.957561    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/532cf3a3-18af-48ac-8df2-ce52f16f98ea-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:11.958000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:11.957569    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/532cf3a3-18af-48ac-8df2-ce52f16f98ea-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:12.042511 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.042481    4925 generic.go:356] "Generic (PLEG): container finished" podID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" containerID="cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4" exitCode=0
Oct 30 19:16:12.042633 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.042534    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"
Oct 30 19:16:12.042633 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.042579    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" event={"ID":"532cf3a3-18af-48ac-8df2-ce52f16f98ea","Type":"ContainerDied","Data":"cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4"}
Oct 30 19:16:12.042750 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.042637    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt" event={"ID":"532cf3a3-18af-48ac-8df2-ce52f16f98ea","Type":"ContainerDied","Data":"9a742c0ac2606f538214325454045185c18ac6bd1aeed3dc220dff706b58a81a"}
Oct 30 19:16:12.042750 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.042663    4925 scope.go:122] "RemoveContainer" containerID="cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4"
Oct 30 19:16:12.054514 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.054496    4925 scope.go:122] "RemoveContainer" containerID="cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4"
Oct 30 19:16:12.054815 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:12.054797    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4\": container with ID starting with cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4 not found: ID does not exist" containerID="cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4"
Oct 30 19:16:12.054870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.054828    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4"} err="failed to get container status \"cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4\": rpc error: code = NotFound desc = could not find container \"cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4\": container with ID starting with cc4727e5af1783aa57f89f18fd0bd13270de6999bbc4eaedacc3758d1d8e9ad4 not found: ID does not exist"
Oct 30 19:16:12.072014 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.071990    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"]
Oct 30 19:16:12.076729 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.076703    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-4a477d-openshift-default-66bfdb8649-49dwt"]
Oct 30 19:16:12.177615 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.177572    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:12.423662 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.423634    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" path="/var/lib/kubelet/pods/532cf3a3-18af-48ac-8df2-ce52f16f98ea/volumes"
Oct 30 19:16:12.424036 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.424023    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82c8263f-5c5f-4cce-adcb-8e15e22a8158" path="/var/lib/kubelet/pods/82c8263f-5c5f-4cce-adcb-8e15e22a8158/volumes"
Oct 30 19:16:12.908175 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.902344    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-9de872-56b8496dc7-h4g87" podUID="bfd0603c-3ee0-476c-916c-e9c4f49b1077" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.55:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:16:12.928962 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:12.928917    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:13.189214 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:13.181942    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:13.307417 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:13.307379    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:16:13.332415 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:13.332376    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"]
Oct 30 19:16:13.406122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:13.406037    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:13.605691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:13.605599    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:13.982066 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:13.982032    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:14.882120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:14.876704    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:15.597124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:15.590083    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:16.178735 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:16.176534    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf" podUID="c71c1a5f-6d34-4b5b-854c-ae1577419337" containerName="payload-pre-processing" containerID="cri-o://1025e4ef46b2956f0245f984392a9f3f892baf4869f90527f0dc58e9b37d8a7a" gracePeriod=25
Oct 30 19:16:16.179621 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:16.178951    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:16.857701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:16.857662    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:16.985812 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:16.985776    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"]
Oct 30 19:16:17.006600 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:17.006533    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"]
Oct 30 19:16:17.166061 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:17.166018    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:17.372050 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:17.372018    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:17.604170 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:17.601488    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"]
Oct 30 19:16:17.604170 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:17.601722    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" containerName="istio-proxy" containerID="cri-o://82a9fd74b11e9bb36b6752a6556899c390195f40a369b468a1f36fee0bcfc94a" gracePeriod=30
Oct 30 19:16:17.616271 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:17.616241    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:18.108075 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:18.108039    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:18.333489 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:18.333449    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95" podUID="c83546ba-7aa3-4999-a888-f317b440593b" containerName="payload-processing" containerID="cri-o://4a3b38c760ff1fb7704209248adec77a3b757f7f6acc5825a313350674f9f5ad" gracePeriod=25
Oct 30 19:16:18.958256 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:18.958220    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:19.575398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:19.575363    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:19.776122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:19.771746    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:19.967133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:19.966882    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:20.165122 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:20.164942    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:20.363172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:20.362251    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:20.581780 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:20.581742    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:20.730583 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:20.730548    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"]
Oct 30 19:16:20.730822 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:20.730795    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8" podUID="93cec944-2c6d-4e13-8fad-c663bd7934ae" containerName="maas-api" containerID="cri-o://d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711" gracePeriod=30
Oct 30 19:16:20.870446 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:20.870409    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:21.208006 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.207983    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:16:21.298687 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.298649    4925 generic.go:356] "Generic (PLEG): container finished" podID="93cec944-2c6d-4e13-8fad-c663bd7934ae" containerID="d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711" exitCode=0
Oct 30 19:16:21.298893 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.298766    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"
Oct 30 19:16:21.298970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.298779    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8" event={"ID":"93cec944-2c6d-4e13-8fad-c663bd7934ae","Type":"ContainerDied","Data":"d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711"}
Oct 30 19:16:21.299040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.299022    4925 scope.go:122] "RemoveContainer" containerID="d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711"
Oct 30 19:16:21.299140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.299025    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8" event={"ID":"93cec944-2c6d-4e13-8fad-c663bd7934ae","Type":"ContainerDied","Data":"cc684cff7eb84fa05c55c9d9e263517dc5ef052d3bf2ff12c3f22ab127867acc"}
Oct 30 19:16:21.303282 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.303259    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-maas-api-tls\") pod \"93cec944-2c6d-4e13-8fad-c663bd7934ae\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") "
Oct 30 19:16:21.303488 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.303418    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93cec944-2c6d-4e13-8fad-c663bd7934ae-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93cec944-2c6d-4e13-8fad-c663bd7934ae-openshift-service-ca\") pod \"93cec944-2c6d-4e13-8fad-c663bd7934ae\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") "
Oct 30 19:16:21.303569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.303545    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93cec944-2c6d-4e13-8fad-c663bd7934ae-kube-api-access-q2tv5\" (UniqueName: \"kubernetes.io/projected/93cec944-2c6d-4e13-8fad-c663bd7934ae-kube-api-access-q2tv5\") pod \"93cec944-2c6d-4e13-8fad-c663bd7934ae\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") "
Oct 30 19:16:21.303631 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.303579    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-metrics-certs\" (UniqueName: \"kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-metrics-certs\") pod \"93cec944-2c6d-4e13-8fad-c663bd7934ae\" (UID: \"93cec944-2c6d-4e13-8fad-c663bd7934ae\") "
Oct 30 19:16:21.304020 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.303959    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93cec944-2c6d-4e13-8fad-c663bd7934ae-openshift-service-ca" pod "93cec944-2c6d-4e13-8fad-c663bd7934ae" (UID: "93cec944-2c6d-4e13-8fad-c663bd7934ae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:21.304257 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.304229    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93cec944-2c6d-4e13-8fad-c663bd7934ae-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:21.306318 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.305580    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:21.306922 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.306897    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93cec944-2c6d-4e13-8fad-c663bd7934ae-kube-api-access-q2tv5" pod "93cec944-2c6d-4e13-8fad-c663bd7934ae" (UID: "93cec944-2c6d-4e13-8fad-c663bd7934ae"). InnerVolumeSpecName "kube-api-access-q2tv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:21.308436 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.308407    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-metrics-certs" pod "93cec944-2c6d-4e13-8fad-c663bd7934ae" (UID: "93cec944-2c6d-4e13-8fad-c663bd7934ae"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:21.309480 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.309459    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-maas-api-tls" pod "93cec944-2c6d-4e13-8fad-c663bd7934ae" (UID: "93cec944-2c6d-4e13-8fad-c663bd7934ae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:21.312443 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.312387    4925 scope.go:122] "RemoveContainer" containerID="d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711"
Oct 30 19:16:21.312796 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:21.312770    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711\": container with ID starting with d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711 not found: ID does not exist" containerID="d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711"
Oct 30 19:16:21.312901 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.312800    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711"} err="failed to get container status \"d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711\": rpc error: code = NotFound desc = could not find container \"d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711\": container with ID starting with d0b3c885d085212d0efeda093a909f3b312897c0a29c1fc9d52f422d72e24711 not found: ID does not exist"
Oct 30 19:16:21.411338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.409785    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2tv5\" (UniqueName: \"kubernetes.io/projected/93cec944-2c6d-4e13-8fad-c663bd7934ae-kube-api-access-q2tv5\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:21.411338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.409828    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:21.411338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.409842    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93cec944-2c6d-4e13-8fad-c663bd7934ae-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:21.506124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.506015    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:21.625321 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.625292    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"]
Oct 30 19:16:21.629731 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.629701    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-a1d55d-778bd55fc9-96bz8"]
Oct 30 19:16:21.819543 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.819458    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:21.965675 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.965641    4925 patch_prober.go:29] interesting pod/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:16:21.965823 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.965711    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:16:21.986958 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:21.986900    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76" podUID="cded1de5-2ce0-454c-8426-22558fd26da6" containerName="payload-processing" containerID="cri-o://9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3" gracePeriod=25
Oct 30 19:16:22.007516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.007475    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9" podUID="0b353a66-fb6e-418d-9538-bf279927ebfc" containerName="payload-pre-processing" containerID="cri-o://3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706" gracePeriod=25
Oct 30 19:16:22.302517 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.302489    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 19:16:22.408162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.407958    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:22.428140 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.428084    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93cec944-2c6d-4e13-8fad-c663bd7934ae" path="/var/lib/kubelet/pods/93cec944-2c6d-4e13-8fad-c663bd7934ae/volumes"
Oct 30 19:16:22.441296 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.441276    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 19:16:22.624733 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.624649    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:22.916753 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.916715    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 19:16:22.921182 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.921089    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95" podUID="c83546ba-7aa3-4999-a888-f317b440593b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:22.927410 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.927378    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-podinfo\") pod \"97223b58-c28a-4e9b-b84c-dbd10acb606e\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") "
Oct 30 19:16:22.927538 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.927432    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-token\" (UniqueName: \"kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-token\") pod \"97223b58-c28a-4e9b-b84c-dbd10acb606e\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") "
Oct 30 19:16:22.927538 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.927470    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-certs\") pod \"97223b58-c28a-4e9b-b84c-dbd10acb606e\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") "
Oct 30 19:16:22.927538 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.927501    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-kube-api-access-65h8f\" (UniqueName: \"kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-kube-api-access-65h8f\") pod \"97223b58-c28a-4e9b-b84c-dbd10acb606e\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") "
Oct 30 19:16:22.927732 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.927575    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97223b58-c28a-4e9b-b84c-dbd10acb606e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97223b58-c28a-4e9b-b84c-dbd10acb606e-istiod-ca-cert\") pod \"97223b58-c28a-4e9b-b84c-dbd10acb606e\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") "
Oct 30 19:16:22.927732 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.927652    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-data\") pod \"97223b58-c28a-4e9b-b84c-dbd10acb606e\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") "
Oct 30 19:16:22.927732 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.927683    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-socket\") pod \"97223b58-c28a-4e9b-b84c-dbd10acb606e\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") "
Oct 30 19:16:22.927882 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.927766    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-credential-socket\") pod \"97223b58-c28a-4e9b-b84c-dbd10acb606e\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") "
Oct 30 19:16:22.927882 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.927817    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-envoy\") pod \"97223b58-c28a-4e9b-b84c-dbd10acb606e\" (UID: \"97223b58-c28a-4e9b-b84c-dbd10acb606e\") "
Oct 30 19:16:22.928507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.928294    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-data" pod "97223b58-c28a-4e9b-b84c-dbd10acb606e" (UID: "97223b58-c28a-4e9b-b84c-dbd10acb606e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:22.928507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.928387    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97223b58-c28a-4e9b-b84c-dbd10acb606e-istiod-ca-cert" pod "97223b58-c28a-4e9b-b84c-dbd10acb606e" (UID: "97223b58-c28a-4e9b-b84c-dbd10acb606e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:22.928507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.928464    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-socket" pod "97223b58-c28a-4e9b-b84c-dbd10acb606e" (UID: "97223b58-c28a-4e9b-b84c-dbd10acb606e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:22.928780 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.928628    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97223b58-c28a-4e9b-b84c-dbd10acb606e-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:22.928780 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.928652    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:22.928780 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.928668    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:22.928780 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.928746    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-certs" pod "97223b58-c28a-4e9b-b84c-dbd10acb606e" (UID: "97223b58-c28a-4e9b-b84c-dbd10acb606e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:22.929454 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.929418    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-credential-socket" pod "97223b58-c28a-4e9b-b84c-dbd10acb606e" (UID: "97223b58-c28a-4e9b-b84c-dbd10acb606e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:22.930392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.930350    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-podinfo" pod "97223b58-c28a-4e9b-b84c-dbd10acb606e" (UID: "97223b58-c28a-4e9b-b84c-dbd10acb606e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:16:22.934409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.934265    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-token" pod "97223b58-c28a-4e9b-b84c-dbd10acb606e" (UID: "97223b58-c28a-4e9b-b84c-dbd10acb606e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:22.934533 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.934433    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-kube-api-access-65h8f" pod "97223b58-c28a-4e9b-b84c-dbd10acb606e" (UID: "97223b58-c28a-4e9b-b84c-dbd10acb606e"). InnerVolumeSpecName "kube-api-access-65h8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:22.934958 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.934923    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-envoy" pod "97223b58-c28a-4e9b-b84c-dbd10acb606e" (UID: "97223b58-c28a-4e9b-b84c-dbd10acb606e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:22.939372 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:22.939192    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main" containerID="cri-o://92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286" gracePeriod=45
Oct 30 19:16:23.030217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.030184    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:23.030217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.030220    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:23.030403 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.030237    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:23.030403 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.030251    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:23.030403 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.030264    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97223b58-c28a-4e9b-b84c-dbd10acb606e-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:23.030403 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.030278    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65h8f\" (UniqueName: \"kubernetes.io/projected/97223b58-c28a-4e9b-b84c-dbd10acb606e-kube-api-access-65h8f\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:23.223808 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.223740    4925 scope.go:122] "RemoveContainer" containerID="82a9fd74b11e9bb36b6752a6556899c390195f40a369b468a1f36fee0bcfc94a"
Oct 30 19:16:23.268974 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.263314    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 19:16:23.282147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.272708    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:23.325241 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.324507    4925 generic.go:356] "Generic (PLEG): container finished" podID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerID="92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286" exitCode=0
Oct 30 19:16:23.325241 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.324677    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" event={"ID":"4d4e5708-1d38-4db4-baef-f5863d72c2d1","Type":"ContainerDied","Data":"92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286"}
Oct 30 19:16:23.325241 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.324707    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4" event={"ID":"4d4e5708-1d38-4db4-baef-f5863d72c2d1","Type":"ContainerDied","Data":"d45c8b0652d63fb08a92dfb978eab37b95df4f69ec135a83fa431e39a0a805c4"}
Oct 30 19:16:23.325241 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.324729    4925 scope.go:122] "RemoveContainer" containerID="92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286"
Oct 30 19:16:23.325241 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.324831    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"
Oct 30 19:16:23.326957 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.326933    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" event={"ID":"97223b58-c28a-4e9b-b84c-dbd10acb606e","Type":"ContainerDied","Data":"82a9fd74b11e9bb36b6752a6556899c390195f40a369b468a1f36fee0bcfc94a"}
Oct 30 19:16:23.327078 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.326977    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976" event={"ID":"97223b58-c28a-4e9b-b84c-dbd10acb606e","Type":"ContainerDied","Data":"eb2e5125fe5db945876ef8737de3504eff4c5b26313b18a9070ec2ad10e05919"}
Oct 30 19:16:23.327078 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.327026    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"
Oct 30 19:16:23.333781 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.333752    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tmp-dir\") pod \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") "
Oct 30 19:16:23.333898 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.333844    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d4e5708-1d38-4db4-baef-f5863d72c2d1-kube-api-access-btk7q\" (UniqueName: \"kubernetes.io/projected/4d4e5708-1d38-4db4-baef-f5863d72c2d1-kube-api-access-btk7q\") pod \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") "
Oct 30 19:16:23.333970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.333950    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-model-cache\") pod \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") "
Oct 30 19:16:23.334026 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.334015    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-home\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-home\") pod \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") "
Oct 30 19:16:23.334118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.334060    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tls-certs\" (UniqueName: \"kubernetes.io/secret/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tls-certs\") pod \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") "
Oct 30 19:16:23.335959 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.334216    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-dshm\") pod \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\" (UID: \"4d4e5708-1d38-4db4-baef-f5863d72c2d1\") "
Oct 30 19:16:23.337873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.337846    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tmp-dir" pod "4d4e5708-1d38-4db4-baef-f5863d72c2d1" (UID: "4d4e5708-1d38-4db4-baef-f5863d72c2d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:23.338129 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.338108    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-home" pod "4d4e5708-1d38-4db4-baef-f5863d72c2d1" (UID: "4d4e5708-1d38-4db4-baef-f5863d72c2d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:23.339159 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.338372    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-model-cache" pod "4d4e5708-1d38-4db4-baef-f5863d72c2d1" (UID: "4d4e5708-1d38-4db4-baef-f5863d72c2d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:23.339159 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.338542    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-dshm" pod "4d4e5708-1d38-4db4-baef-f5863d72c2d1" (UID: "4d4e5708-1d38-4db4-baef-f5863d72c2d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:23.342812 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.340511    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tls-certs" pod "4d4e5708-1d38-4db4-baef-f5863d72c2d1" (UID: "4d4e5708-1d38-4db4-baef-f5863d72c2d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:23.342812 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.340558    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d4e5708-1d38-4db4-baef-f5863d72c2d1-kube-api-access-btk7q" pod "4d4e5708-1d38-4db4-baef-f5863d72c2d1" (UID: "4d4e5708-1d38-4db4-baef-f5863d72c2d1"). InnerVolumeSpecName "kube-api-access-btk7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:23.354559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.354536    4925 scope.go:122] "RemoveContainer" containerID="92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286"
Oct 30 19:16:23.355834 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:23.355462    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286\": container with ID starting with 92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286 not found: ID does not exist" containerID="92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286"
Oct 30 19:16:23.355834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.355498    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286"} err="failed to get container status \"92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286\": rpc error: code = NotFound desc = could not find container \"92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286\": container with ID starting with 92011a293ce7620efd63aa734865805b4aa1c55a1860b52b6bb877487876d286 not found: ID does not exist"
Oct 30 19:16:23.374380 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.370874    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"]
Oct 30 19:16:23.376194 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.376172    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-9de872-openshift-default-77cf8df555-8g976"]
Oct 30 19:16:23.436596 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.436570    4925 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-dshm\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:23.436685 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.436602    4925 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tmp-dir\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:23.436685 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.436627    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btk7q\" (UniqueName: \"kubernetes.io/projected/4d4e5708-1d38-4db4-baef-f5863d72c2d1-kube-api-access-btk7q\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:23.436685 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.436641    4925 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-model-cache\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:23.436685 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.436651    4925 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d4e5708-1d38-4db4-baef-f5863d72c2d1-home\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:23.436685 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.436660    4925 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d4e5708-1d38-4db4-baef-f5863d72c2d1-tls-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:23.648774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.648742    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"]
Oct 30 19:16:23.651414 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.651391    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-0458df/distinct-2-w4-0458df-kserve-cddd569df-2szt4"]
Oct 30 19:16:23.864394 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:23.864287    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:24.051719 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:24.050029    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:24.422752 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:24.422711    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" path="/var/lib/kubelet/pods/4d4e5708-1d38-4db4-baef-f5863d72c2d1/volumes"
Oct 30 19:16:24.423397 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:24.423375    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" path="/var/lib/kubelet/pods/97223b58-c28a-4e9b-b84c-dbd10acb606e/volumes"
Oct 30 19:16:24.873604 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:24.873494    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:25.303214 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:25.303160    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf" podUID="c71c1a5f-6d34-4b5b-854c-ae1577419337" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:25.602701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:25.602613    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:26.325118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:26.320452    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:27.106123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:27.098672    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:27.386229 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:27.386077    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:27.574846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:27.574813    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"]
Oct 30 19:16:27.575069 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:27.575042    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2" podUID="a4d3263e-e6be-4c1e-9700-b654cedc6258" containerName="istio-proxy" containerID="cri-o://f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1" gracePeriod=30
Oct 30 19:16:27.608004 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:27.607970    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:27.804933 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:27.804722    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:28.041435 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:28.041400    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:28.219020 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:28.218988    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:28.982885 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:28.982852    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:29.030011 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:29.029979    4925 patch_prober.go:29] interesting pod/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:16:29.030215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:29.030041    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2" podUID="a4d3263e-e6be-4c1e-9700-b654cedc6258" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:16:29.245340 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:29.245246    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:29.428723 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:29.428682    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:29.518249 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:29.518161    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"]
Oct 30 19:16:29.518440 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:29.518416    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy" containerID="cri-o://49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9" gracePeriod=30
Oct 30 19:16:29.661120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:29.660246    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76" podUID="cded1de5-2ce0-454c-8426-22558fd26da6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:29.665154 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:29.663175    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9" podUID="0b353a66-fb6e-418d-9538-bf279927ebfc" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:29.757078 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:29.757002    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:29.968393 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:29.968355    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:30.775118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:30.772927    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"]
Oct 30 19:16:30.795166 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:30.795133    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"]
Oct 30 19:16:30.835352 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:30.835323    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:31.127454 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:31.127338    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:31.698229 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:31.698194    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:31.911681 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:31.911645    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:32.125738 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:32.125610    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:32.502962 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:32.502924    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:32.782817 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:32.782726    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:32.913793 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:32.913767    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:16:32.921211 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:32.921183    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95" podUID="c83546ba-7aa3-4999-a888-f317b440593b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:33.021295 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.021263    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"]
Oct 30 19:16:33.048611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.048529    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-podinfo\") pod \"a4d3263e-e6be-4c1e-9700-b654cedc6258\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") "
Oct 30 19:16:33.048611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.048572    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4d3263e-e6be-4c1e-9700-b654cedc6258-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4d3263e-e6be-4c1e-9700-b654cedc6258-istiod-ca-cert\") pod \"a4d3263e-e6be-4c1e-9700-b654cedc6258\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") "
Oct 30 19:16:33.048611 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.048607    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-envoy\") pod \"a4d3263e-e6be-4c1e-9700-b654cedc6258\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") "
Oct 30 19:16:33.049144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.048762    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-socket\") pod \"a4d3263e-e6be-4c1e-9700-b654cedc6258\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") "
Oct 30 19:16:33.049144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.048875    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-kube-api-access-tzt46\" (UniqueName: \"kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-kube-api-access-tzt46\") pod \"a4d3263e-e6be-4c1e-9700-b654cedc6258\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") "
Oct 30 19:16:33.049144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.048945    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4d3263e-e6be-4c1e-9700-b654cedc6258-istiod-ca-cert" pod "a4d3263e-e6be-4c1e-9700-b654cedc6258" (UID: "a4d3263e-e6be-4c1e-9700-b654cedc6258"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:33.049501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.049179    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-socket" pod "a4d3263e-e6be-4c1e-9700-b654cedc6258" (UID: "a4d3263e-e6be-4c1e-9700-b654cedc6258"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:33.049501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.049387    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-credential-socket\") pod \"a4d3263e-e6be-4c1e-9700-b654cedc6258\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") "
Oct 30 19:16:33.049501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.049477    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-certs\") pod \"a4d3263e-e6be-4c1e-9700-b654cedc6258\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") "
Oct 30 19:16:33.049658 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.049506    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-token\" (UniqueName: \"kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-token\") pod \"a4d3263e-e6be-4c1e-9700-b654cedc6258\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") "
Oct 30 19:16:33.049658 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.049593    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-data\") pod \"a4d3263e-e6be-4c1e-9700-b654cedc6258\" (UID: \"a4d3263e-e6be-4c1e-9700-b654cedc6258\") "
Oct 30 19:16:33.049772 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.049657    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-credential-socket" pod "a4d3263e-e6be-4c1e-9700-b654cedc6258" (UID: "a4d3263e-e6be-4c1e-9700-b654cedc6258"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:33.050394 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.050365    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.050513 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.050398    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4d3263e-e6be-4c1e-9700-b654cedc6258-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.050513 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.050413    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.050663 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.050639    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-data" pod "a4d3263e-e6be-4c1e-9700-b654cedc6258" (UID: "a4d3263e-e6be-4c1e-9700-b654cedc6258"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:33.050818 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.050790    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-certs" pod "a4d3263e-e6be-4c1e-9700-b654cedc6258" (UID: "a4d3263e-e6be-4c1e-9700-b654cedc6258"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:33.051809 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.051548    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-podinfo" pod "a4d3263e-e6be-4c1e-9700-b654cedc6258" (UID: "a4d3263e-e6be-4c1e-9700-b654cedc6258"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:16:33.051809 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.051787    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-kube-api-access-tzt46" pod "a4d3263e-e6be-4c1e-9700-b654cedc6258" (UID: "a4d3263e-e6be-4c1e-9700-b654cedc6258"). InnerVolumeSpecName "kube-api-access-tzt46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:33.052521 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.052502    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-token" pod "a4d3263e-e6be-4c1e-9700-b654cedc6258" (UID: "a4d3263e-e6be-4c1e-9700-b654cedc6258"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:33.053270 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.053247    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-envoy" pod "a4d3263e-e6be-4c1e-9700-b654cedc6258" (UID: "a4d3263e-e6be-4c1e-9700-b654cedc6258"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:33.151930 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.151897    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzt46\" (UniqueName: \"kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-kube-api-access-tzt46\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.152077 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.151935    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.152077 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.151951    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.152077 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.151966    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.152077 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.151979    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.152077 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.151992    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4d3263e-e6be-4c1e-9700-b654cedc6258-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.278957 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.278908    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:33.321243 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.319422    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"]
Oct 30 19:16:33.321243 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.319625    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj" podUID="dfa60598-c22f-4caa-a3dc-e3092197742f" containerName="maas-api" containerID="cri-o://48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4" gracePeriod=30
Oct 30 19:16:33.419040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.419008    4925 generic.go:356] "Generic (PLEG): container finished" podID="a4d3263e-e6be-4c1e-9700-b654cedc6258" containerID="f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1" exitCode=0
Oct 30 19:16:33.419220 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.419050    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"
Oct 30 19:16:33.419220 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.419122    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2" event={"ID":"a4d3263e-e6be-4c1e-9700-b654cedc6258","Type":"ContainerDied","Data":"f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1"}
Oct 30 19:16:33.419220 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.419171    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2" event={"ID":"a4d3263e-e6be-4c1e-9700-b654cedc6258","Type":"ContainerDied","Data":"c4d51a2081dd4f243a0f4eaf80c8d2d438ff029f26fe7b6e9348103518b71652"}
Oct 30 19:16:33.419220 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.419193    4925 scope.go:122] "RemoveContainer" containerID="f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1"
Oct 30 19:16:33.433749 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.433726    4925 scope.go:122] "RemoveContainer" containerID="f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1"
Oct 30 19:16:33.434054 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:33.434033    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1\": container with ID starting with f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1 not found: ID does not exist" containerID="f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1"
Oct 30 19:16:33.434156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.434061    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1"} err="failed to get container status \"f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1\": rpc error: code = NotFound desc = could not find container \"f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1\": container with ID starting with f856cf4e00b96f7275639ca73f202b3b05847b798e4261a71e68a7fb5c89f5e1 not found: ID does not exist"
Oct 30 19:16:33.443580 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.443554    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"]
Oct 30 19:16:33.446734 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.446697    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-2d1c71-openshift-default-5dc796b975-89xf2"]
Oct 30 19:16:33.453144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.452969    4925 patch_prober.go:29] interesting pod/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:16:33.453144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.453020    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:16:33.491526 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.491465    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:33.696115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.695956    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:33.736567 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.736540    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 19:16:33.749198 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.749156    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-maas-api-tls\") pod \"dfa60598-c22f-4caa-a3dc-e3092197742f\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") "
Oct 30 19:16:33.749362 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.749299    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dfa60598-c22f-4caa-a3dc-e3092197742f-kube-api-access-s48pg\" (UniqueName: \"kubernetes.io/projected/dfa60598-c22f-4caa-a3dc-e3092197742f-kube-api-access-s48pg\") pod \"dfa60598-c22f-4caa-a3dc-e3092197742f\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") "
Oct 30 19:16:33.749440 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.749426    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dfa60598-c22f-4caa-a3dc-e3092197742f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfa60598-c22f-4caa-a3dc-e3092197742f-openshift-service-ca\") pod \"dfa60598-c22f-4caa-a3dc-e3092197742f\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") "
Oct 30 19:16:33.749507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.749492    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-metrics-certs\") pod \"dfa60598-c22f-4caa-a3dc-e3092197742f\" (UID: \"dfa60598-c22f-4caa-a3dc-e3092197742f\") "
Oct 30 19:16:33.749851 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.749820    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfa60598-c22f-4caa-a3dc-e3092197742f-openshift-service-ca" pod "dfa60598-c22f-4caa-a3dc-e3092197742f" (UID: "dfa60598-c22f-4caa-a3dc-e3092197742f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:33.750313 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.750285    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dfa60598-c22f-4caa-a3dc-e3092197742f-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.752366 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.752331    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa60598-c22f-4caa-a3dc-e3092197742f-kube-api-access-s48pg" pod "dfa60598-c22f-4caa-a3dc-e3092197742f" (UID: "dfa60598-c22f-4caa-a3dc-e3092197742f"). InnerVolumeSpecName "kube-api-access-s48pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:33.752673 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.752645    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-metrics-certs" pod "dfa60598-c22f-4caa-a3dc-e3092197742f" (UID: "dfa60598-c22f-4caa-a3dc-e3092197742f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:33.754231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.754210    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-maas-api-tls" pod "dfa60598-c22f-4caa-a3dc-e3092197742f" (UID: "dfa60598-c22f-4caa-a3dc-e3092197742f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:33.851636 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.851606    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s48pg\" (UniqueName: \"kubernetes.io/projected/dfa60598-c22f-4caa-a3dc-e3092197742f-kube-api-access-s48pg\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.851636 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.851637    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.851830 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.851648    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dfa60598-c22f-4caa-a3dc-e3092197742f-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:33.945081 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:33.933242    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:34.135419 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.135379    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:34.365564 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.365476    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:34.425994 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.425958    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4d3263e-e6be-4c1e-9700-b654cedc6258" path="/var/lib/kubelet/pods/a4d3263e-e6be-4c1e-9700-b654cedc6258/volumes"
Oct 30 19:16:34.430754 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.430725    4925 generic.go:356] "Generic (PLEG): container finished" podID="dfa60598-c22f-4caa-a3dc-e3092197742f" containerID="48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4" exitCode=0
Oct 30 19:16:34.430930 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.430771    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"
Oct 30 19:16:34.430930 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.430812    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj" event={"ID":"dfa60598-c22f-4caa-a3dc-e3092197742f","Type":"ContainerDied","Data":"48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4"}
Oct 30 19:16:34.430930 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.430864    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj" event={"ID":"dfa60598-c22f-4caa-a3dc-e3092197742f","Type":"ContainerDied","Data":"05c8ff6f7070a5cedf7376ac91b5d758376df802ab1d40eda00e57e29981f01b"}
Oct 30 19:16:34.430930 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.430894    4925 scope.go:122] "RemoveContainer" containerID="48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4"
Oct 30 19:16:34.443078 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.443058    4925 scope.go:122] "RemoveContainer" containerID="48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4"
Oct 30 19:16:34.443421 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:34.443395    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4\": container with ID starting with 48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4 not found: ID does not exist" containerID="48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4"
Oct 30 19:16:34.443532 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.443427    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4"} err="failed to get container status \"48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4\": rpc error: code = NotFound desc = could not find container \"48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4\": container with ID starting with 48b8a0c220abe95f126c960a75cf01ee496cbdf1bb3431656458e1788ffc44e4 not found: ID does not exist"
Oct 30 19:16:34.455515 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.455490    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"]
Oct 30 19:16:34.459940 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.459917    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-0458df-7cf96d7d4c-qppqj"]
Oct 30 19:16:34.836970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.836935    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:16:34.839570 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.839541    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:34.870087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.868021    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-envoy\") pod \"f5760e51-b421-4e2b-8896-e61d63ef0e18\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") "
Oct 30 19:16:34.870087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.868163    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-data\") pod \"f5760e51-b421-4e2b-8896-e61d63ef0e18\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") "
Oct 30 19:16:34.870087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.868200    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-credential-socket\") pod \"f5760e51-b421-4e2b-8896-e61d63ef0e18\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") "
Oct 30 19:16:34.870087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.868236    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-kube-api-access-sb6qb\" (UniqueName: \"kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-kube-api-access-sb6qb\") pod \"f5760e51-b421-4e2b-8896-e61d63ef0e18\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") "
Oct 30 19:16:34.870087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.868304    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-socket\") pod \"f5760e51-b421-4e2b-8896-e61d63ef0e18\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") "
Oct 30 19:16:34.870087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.868349    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-podinfo\") pod \"f5760e51-b421-4e2b-8896-e61d63ef0e18\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") "
Oct 30 19:16:34.870087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.868369    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-certs\") pod \"f5760e51-b421-4e2b-8896-e61d63ef0e18\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") "
Oct 30 19:16:34.870087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.868419    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-token\" (UniqueName: \"kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-token\") pod \"f5760e51-b421-4e2b-8896-e61d63ef0e18\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") "
Oct 30 19:16:34.870087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.868454    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5760e51-b421-4e2b-8896-e61d63ef0e18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5760e51-b421-4e2b-8896-e61d63ef0e18-istiod-ca-cert\") pod \"f5760e51-b421-4e2b-8896-e61d63ef0e18\" (UID: \"f5760e51-b421-4e2b-8896-e61d63ef0e18\") "
Oct 30 19:16:34.870087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.869175    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5760e51-b421-4e2b-8896-e61d63ef0e18-istiod-ca-cert" pod "f5760e51-b421-4e2b-8896-e61d63ef0e18" (UID: "f5760e51-b421-4e2b-8896-e61d63ef0e18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:34.870087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.869708    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-certs" pod "f5760e51-b421-4e2b-8896-e61d63ef0e18" (UID: "f5760e51-b421-4e2b-8896-e61d63ef0e18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:34.870737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.870529    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-data" pod "f5760e51-b421-4e2b-8896-e61d63ef0e18" (UID: "f5760e51-b421-4e2b-8896-e61d63ef0e18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:34.870737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.870583    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-credential-socket" pod "f5760e51-b421-4e2b-8896-e61d63ef0e18" (UID: "f5760e51-b421-4e2b-8896-e61d63ef0e18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:34.870814 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.870790    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-socket" pod "f5760e51-b421-4e2b-8896-e61d63ef0e18" (UID: "f5760e51-b421-4e2b-8896-e61d63ef0e18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:34.872981 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.872953    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-kube-api-access-sb6qb" pod "f5760e51-b421-4e2b-8896-e61d63ef0e18" (UID: "f5760e51-b421-4e2b-8896-e61d63ef0e18"). InnerVolumeSpecName "kube-api-access-sb6qb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:34.873262 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.873239    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-envoy" pod "f5760e51-b421-4e2b-8896-e61d63ef0e18" (UID: "f5760e51-b421-4e2b-8896-e61d63ef0e18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:34.877825 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.877786    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-token" pod "f5760e51-b421-4e2b-8896-e61d63ef0e18" (UID: "f5760e51-b421-4e2b-8896-e61d63ef0e18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:34.879028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.879004    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-podinfo" pod "f5760e51-b421-4e2b-8896-e61d63ef0e18" (UID: "f5760e51-b421-4e2b-8896-e61d63ef0e18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:16:34.970627 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.970601    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:34.970627 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.970625    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:34.970970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.970636    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:34.970970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.970645    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5760e51-b421-4e2b-8896-e61d63ef0e18-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:34.970970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.970654    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:34.970970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.970662    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:34.970970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.970670    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:34.970970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.970678    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb6qb\" (UniqueName: \"kubernetes.io/projected/f5760e51-b421-4e2b-8896-e61d63ef0e18-kube-api-access-sb6qb\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:34.970970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:34.970687    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5760e51-b421-4e2b-8896-e61d63ef0e18-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:35.303161 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.303115    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf" podUID="c71c1a5f-6d34-4b5b-854c-ae1577419337" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:35.439687 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.439659    4925 generic.go:356] "Generic (PLEG): container finished" podID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerID="49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9" exitCode=0
Oct 30 19:16:35.439687 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.439691    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" event={"ID":"f5760e51-b421-4e2b-8896-e61d63ef0e18","Type":"ContainerDied","Data":"49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9"}
Oct 30 19:16:35.439873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.439699    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"
Oct 30 19:16:35.439873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.439724    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v" event={"ID":"f5760e51-b421-4e2b-8896-e61d63ef0e18","Type":"ContainerDied","Data":"52fbb4c86a958e969db5d5d6fc887858a2c93b3c20006e5e639a793d3cddb063"}
Oct 30 19:16:35.439873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.439744    4925 scope.go:122] "RemoveContainer" containerID="49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9"
Oct 30 19:16:35.450840 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.450822    4925 scope.go:122] "RemoveContainer" containerID="5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8"
Oct 30 19:16:35.462053 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.462030    4925 scope.go:122] "RemoveContainer" containerID="49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9"
Oct 30 19:16:35.462511 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:35.462487    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9\": container with ID starting with 49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9 not found: ID does not exist" containerID="49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9"
Oct 30 19:16:35.462673 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.462645    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9"} err="failed to get container status \"49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9\": rpc error: code = NotFound desc = could not find container \"49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9\": container with ID starting with 49b55df1f3a4ae9869a19bcbd45164e8cb3a0835072a18685499f75ed9497df9 not found: ID does not exist"
Oct 30 19:16:35.462769 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.462759    4925 scope.go:122] "RemoveContainer" containerID="5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8"
Oct 30 19:16:35.463210 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:35.463154    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8\": container with ID starting with 5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8 not found: ID does not exist" containerID="5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8"
Oct 30 19:16:35.463210 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.463188    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8"} err="failed to get container status \"5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8\": rpc error: code = NotFound desc = could not find container \"5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8\": container with ID starting with 5dcc8dda32c80b3082f5ca346f69528dfba00bded7cda99649ff78a580b9ccd8 not found: ID does not exist"
Oct 30 19:16:35.465643 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.465614    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"]
Oct 30 19:16:35.472106 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.472076    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-a1d55d-openshift-default-6646f694ff-8r55v"]
Oct 30 19:16:35.742011 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.741968    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:35.773336 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.773291    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j" podUID="829ee262-f7f8-495c-ba74-1a6e68f79b2f" containerName="payload-processing" containerID="cri-o://463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2" gracePeriod=25
Oct 30 19:16:35.796250 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:35.796211    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk" podUID="52fd9b2d-aef2-4a90-872d-5daa0c44dfd3" containerName="payload-pre-processing" containerID="cri-o://3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9" gracePeriod=25
Oct 30 19:16:36.494121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.489802    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dfa60598-c22f-4caa-a3dc-e3092197742f" path="/var/lib/kubelet/pods/dfa60598-c22f-4caa-a3dc-e3092197742f/volumes"
Oct 30 19:16:36.510871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.497772    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:36.510871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.503470    4925 generic.go:356] "Generic (PLEG): container finished" podID="c71c1a5f-6d34-4b5b-854c-ae1577419337" containerID="1025e4ef46b2956f0245f984392a9f3f892baf4869f90527f0dc58e9b37d8a7a" exitCode=0
Oct 30 19:16:36.510871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.509888    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"
Oct 30 19:16:36.513229 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.512934    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" path="/var/lib/kubelet/pods/f5760e51-b421-4e2b-8896-e61d63ef0e18/volumes"
Oct 30 19:16:36.513484 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.513457    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf" event={"ID":"c71c1a5f-6d34-4b5b-854c-ae1577419337","Type":"ContainerDied","Data":"1025e4ef46b2956f0245f984392a9f3f892baf4869f90527f0dc58e9b37d8a7a"}
Oct 30 19:16:36.513568 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.513527    4925 scope.go:122] "RemoveContainer" containerID="1025e4ef46b2956f0245f984392a9f3f892baf4869f90527f0dc58e9b37d8a7a"
Oct 30 19:16:36.595359 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.595325    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c71c1a5f-6d34-4b5b-854c-ae1577419337-kube-api-access-6hn9q\" (UniqueName: \"kubernetes.io/projected/c71c1a5f-6d34-4b5b-854c-ae1577419337-kube-api-access-6hn9q\") pod \"c71c1a5f-6d34-4b5b-854c-ae1577419337\" (UID: \"c71c1a5f-6d34-4b5b-854c-ae1577419337\") "
Oct 30 19:16:36.595520 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.595368    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c71c1a5f-6d34-4b5b-854c-ae1577419337-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c71c1a5f-6d34-4b5b-854c-ae1577419337-plugins-config-volume\") pod \"c71c1a5f-6d34-4b5b-854c-ae1577419337\" (UID: \"c71c1a5f-6d34-4b5b-854c-ae1577419337\") "
Oct 30 19:16:36.595829 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.595803    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c71c1a5f-6d34-4b5b-854c-ae1577419337-plugins-config-volume" pod "c71c1a5f-6d34-4b5b-854c-ae1577419337" (UID: "c71c1a5f-6d34-4b5b-854c-ae1577419337"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:36.595932 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.595915    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c71c1a5f-6d34-4b5b-854c-ae1577419337-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:36.597742 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.597717    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c71c1a5f-6d34-4b5b-854c-ae1577419337-kube-api-access-6hn9q" pod "c71c1a5f-6d34-4b5b-854c-ae1577419337" (UID: "c71c1a5f-6d34-4b5b-854c-ae1577419337"). InnerVolumeSpecName "kube-api-access-6hn9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:36.697332 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:36.697300    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hn9q\" (UniqueName: \"kubernetes.io/projected/c71c1a5f-6d34-4b5b-854c-ae1577419337-kube-api-access-6hn9q\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:37.324900 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:37.324843    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:37.525474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:37.525448    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"
Oct 30 19:16:37.525865 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:37.525454    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf" event={"ID":"c71c1a5f-6d34-4b5b-854c-ae1577419337","Type":"ContainerDied","Data":"b799175680963112ffc359386d2add468f3529c4bebe60bb4380321e1b32864e"}
Oct 30 19:16:37.551216 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:37.551191    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"]
Oct 30 19:16:37.561537 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:37.561510    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-2d1c71-56bb5877f9-cvwvf"]
Oct 30 19:16:37.998602 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:37.998568    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:38.423579 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.423552    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c71c1a5f-6d34-4b5b-854c-ae1577419337" path="/var/lib/kubelet/pods/c71c1a5f-6d34-4b5b-854c-ae1577419337/volumes"
Oct 30 19:16:38.434136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.434078    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"]
Oct 30 19:16:38.443524 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.443497    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"]
Oct 30 19:16:38.543505 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.543474    4925 generic.go:356] "Generic (PLEG): container finished" podID="c83546ba-7aa3-4999-a888-f317b440593b" containerID="4a3b38c760ff1fb7704209248adec77a3b757f7f6acc5825a313350674f9f5ad" exitCode=0
Oct 30 19:16:38.543916 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.543536    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95" event={"ID":"c83546ba-7aa3-4999-a888-f317b440593b","Type":"ContainerDied","Data":"4a3b38c760ff1fb7704209248adec77a3b757f7f6acc5825a313350674f9f5ad"}
Oct 30 19:16:38.553174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.553143    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk" podUID="52fd9b2d-aef2-4a90-872d-5daa0c44dfd3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:38.556938 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.556914    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j" podUID="829ee262-f7f8-495c-ba74-1a6e68f79b2f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:38.602434 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.602358    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:38.820119 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.814686    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:38.823306 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.823273    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"
Oct 30 19:16:38.856062 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.855973    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c83546ba-7aa3-4999-a888-f317b440593b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c83546ba-7aa3-4999-a888-f317b440593b-plugins-config-volume\") pod \"c83546ba-7aa3-4999-a888-f317b440593b\" (UID: \"c83546ba-7aa3-4999-a888-f317b440593b\") "
Oct 30 19:16:38.857203 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.856084    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c83546ba-7aa3-4999-a888-f317b440593b-kube-api-access-dwvzc\" (UniqueName: \"kubernetes.io/projected/c83546ba-7aa3-4999-a888-f317b440593b-kube-api-access-dwvzc\") pod \"c83546ba-7aa3-4999-a888-f317b440593b\" (UID: \"c83546ba-7aa3-4999-a888-f317b440593b\") "
Oct 30 19:16:38.876164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.868717    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c83546ba-7aa3-4999-a888-f317b440593b-plugins-config-volume" pod "c83546ba-7aa3-4999-a888-f317b440593b" (UID: "c83546ba-7aa3-4999-a888-f317b440593b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:38.876164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.868938    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c83546ba-7aa3-4999-a888-f317b440593b-kube-api-access-dwvzc" pod "c83546ba-7aa3-4999-a888-f317b440593b" (UID: "c83546ba-7aa3-4999-a888-f317b440593b"). InnerVolumeSpecName "kube-api-access-dwvzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:38.958299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.958258    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c83546ba-7aa3-4999-a888-f317b440593b-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:38.958299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:38.958298    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwvzc\" (UniqueName: \"kubernetes.io/projected/c83546ba-7aa3-4999-a888-f317b440593b-kube-api-access-dwvzc\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:39.117858 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:39.116299    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:39.343517 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:39.343482    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:39.551781 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:39.551753    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"
Oct 30 19:16:39.551781 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:39.551764    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95" event={"ID":"c83546ba-7aa3-4999-a888-f317b440593b","Type":"ContainerDied","Data":"d959325347985776bbf24a87451f9a5f213c1f9b8afd0158621081b5adfff50d"}
Oct 30 19:16:39.552320 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:39.551832    4925 scope.go:122] "RemoveContainer" containerID="4a3b38c760ff1fb7704209248adec77a3b757f7f6acc5825a313350674f9f5ad"
Oct 30 19:16:39.581035 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:39.581006    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"]
Oct 30 19:16:39.593427 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:39.593402    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-0458df-8458bc48d8-k4x95"]
Oct 30 19:16:39.627165 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:39.627130    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76" podUID="cded1de5-2ce0-454c-8426-22558fd26da6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:39.627165 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:39.627151    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9" podUID="0b353a66-fb6e-418d-9538-bf279927ebfc" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:39.697231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:39.693285    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:40.043968 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:40.043931    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:40.436552 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:40.436519    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c83546ba-7aa3-4999-a888-f317b440593b" path="/var/lib/kubelet/pods/c83546ba-7aa3-4999-a888-f317b440593b/volumes"
Oct 30 19:16:40.560310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:40.560268    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:40.773323 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:40.773232    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:41.309814 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:41.309774    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:41.641965 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:41.641886    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:41.837339 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:41.837280    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:42.042149 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.038808    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:42.236985 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.236878    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:42.431035 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.431011    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"
Oct 30 19:16:42.434616 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.434597    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"
Oct 30 19:16:42.455657 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.455630    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:42.502620 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.502585    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b353a66-fb6e-418d-9538-bf279927ebfc-kube-api-access-cvvcj\" (UniqueName: \"kubernetes.io/projected/0b353a66-fb6e-418d-9538-bf279927ebfc-kube-api-access-cvvcj\") pod \"0b353a66-fb6e-418d-9538-bf279927ebfc\" (UID: \"0b353a66-fb6e-418d-9538-bf279927ebfc\") "
Oct 30 19:16:42.502782 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.502702    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cded1de5-2ce0-454c-8426-22558fd26da6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cded1de5-2ce0-454c-8426-22558fd26da6-plugins-config-volume\") pod \"cded1de5-2ce0-454c-8426-22558fd26da6\" (UID: \"cded1de5-2ce0-454c-8426-22558fd26da6\") "
Oct 30 19:16:42.502782 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.502759    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b353a66-fb6e-418d-9538-bf279927ebfc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b353a66-fb6e-418d-9538-bf279927ebfc-plugins-config-volume\") pod \"0b353a66-fb6e-418d-9538-bf279927ebfc\" (UID: \"0b353a66-fb6e-418d-9538-bf279927ebfc\") "
Oct 30 19:16:42.502917 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.502904    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cded1de5-2ce0-454c-8426-22558fd26da6-kube-api-access-h59dh\" (UniqueName: \"kubernetes.io/projected/cded1de5-2ce0-454c-8426-22558fd26da6-kube-api-access-h59dh\") pod \"cded1de5-2ce0-454c-8426-22558fd26da6\" (UID: \"cded1de5-2ce0-454c-8426-22558fd26da6\") "
Oct 30 19:16:42.503221 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.503153    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cded1de5-2ce0-454c-8426-22558fd26da6-plugins-config-volume" pod "cded1de5-2ce0-454c-8426-22558fd26da6" (UID: "cded1de5-2ce0-454c-8426-22558fd26da6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:42.503221 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.503208    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b353a66-fb6e-418d-9538-bf279927ebfc-plugins-config-volume" pod "0b353a66-fb6e-418d-9538-bf279927ebfc" (UID: "0b353a66-fb6e-418d-9538-bf279927ebfc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:42.503738 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.503715    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cded1de5-2ce0-454c-8426-22558fd26da6-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:42.503738 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.503742    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b353a66-fb6e-418d-9538-bf279927ebfc-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:42.504961 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.504941    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b353a66-fb6e-418d-9538-bf279927ebfc-kube-api-access-cvvcj" pod "0b353a66-fb6e-418d-9538-bf279927ebfc" (UID: "0b353a66-fb6e-418d-9538-bf279927ebfc"). InnerVolumeSpecName "kube-api-access-cvvcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:42.505247 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.505223    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cded1de5-2ce0-454c-8426-22558fd26da6-kube-api-access-h59dh" pod "cded1de5-2ce0-454c-8426-22558fd26da6" (UID: "cded1de5-2ce0-454c-8426-22558fd26da6"). InnerVolumeSpecName "kube-api-access-h59dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:42.573635 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.573565    4925 generic.go:356] "Generic (PLEG): container finished" podID="0b353a66-fb6e-418d-9538-bf279927ebfc" containerID="3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706" exitCode=0
Oct 30 19:16:42.573635 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.573610    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"
Oct 30 19:16:42.573821 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.573648    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9" event={"ID":"0b353a66-fb6e-418d-9538-bf279927ebfc","Type":"ContainerDied","Data":"3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706"}
Oct 30 19:16:42.573821 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.573697    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9" event={"ID":"0b353a66-fb6e-418d-9538-bf279927ebfc","Type":"ContainerDied","Data":"b5d84391dbaa25e98dd334894cc331c618907f5c9424202bfb761a54b575e8fb"}
Oct 30 19:16:42.573821 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.573719    4925 scope.go:122] "RemoveContainer" containerID="3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706"
Oct 30 19:16:42.579931 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.579374    4925 generic.go:356] "Generic (PLEG): container finished" podID="cded1de5-2ce0-454c-8426-22558fd26da6" containerID="9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3" exitCode=0
Oct 30 19:16:42.579931 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.579546    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76" event={"ID":"cded1de5-2ce0-454c-8426-22558fd26da6","Type":"ContainerDied","Data":"9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3"}
Oct 30 19:16:42.579931 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.579586    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76" event={"ID":"cded1de5-2ce0-454c-8426-22558fd26da6","Type":"ContainerDied","Data":"efeed0c61449eee382c5eeea5b544bcdc6a64f7bacf3b0b9cac422cf46de08da"}
Oct 30 19:16:42.579931 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.579660    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"
Oct 30 19:16:42.589797 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.589680    4925 scope.go:122] "RemoveContainer" containerID="3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706"
Oct 30 19:16:42.590065 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:42.590037    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706\": container with ID starting with 3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706 not found: ID does not exist" containerID="3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706"
Oct 30 19:16:42.590213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.590074    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706"} err="failed to get container status \"3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706\": rpc error: code = NotFound desc = could not find container \"3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706\": container with ID starting with 3854c3225164488b17faaee21315abcc9d6bbd40e19c17462806fe4efd270706 not found: ID does not exist"
Oct 30 19:16:42.590213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.590136    4925 scope.go:122] "RemoveContainer" containerID="9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3"
Oct 30 19:16:42.599399 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.599378    4925 scope.go:122] "RemoveContainer" containerID="9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3"
Oct 30 19:16:42.599657 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:42.599638    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3\": container with ID starting with 9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3 not found: ID does not exist" containerID="9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3"
Oct 30 19:16:42.599727 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.599668    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3"} err="failed to get container status \"9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3\": rpc error: code = NotFound desc = could not find container \"9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3\": container with ID starting with 9e77f6517dce06f4246ab03dcf752315fdd1c293da40a2be4290f757802233a3 not found: ID does not exist"
Oct 30 19:16:42.604598 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.604576    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvvcj\" (UniqueName: \"kubernetes.io/projected/0b353a66-fb6e-418d-9538-bf279927ebfc-kube-api-access-cvvcj\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:42.604598 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.604598    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h59dh\" (UniqueName: \"kubernetes.io/projected/cded1de5-2ce0-454c-8426-22558fd26da6-kube-api-access-h59dh\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:42.607381 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.607358    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"]
Oct 30 19:16:42.609973 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.609949    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-a1d55d-574bc6b555-p7hh9"]
Oct 30 19:16:42.619015 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.618992    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"]
Oct 30 19:16:42.626227 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:42.626205    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-a1d55d-669c459d8-x2x76"]
Oct 30 19:16:43.268784 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:43.268469    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:43.302277 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:43.302203    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:16:43.434559 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:43.434526    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4" podUID="1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07" containerName="payload-processing" containerID="cri-o://3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15" gracePeriod=25
Oct 30 19:16:43.444061 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:43.444029    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz" podUID="9ae0a880-733c-4773-ad10-7239533263df" containerName="payload-pre-processing" containerID="cri-o://972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f" gracePeriod=25
Oct 30 19:16:43.612037 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:43.611943    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:44.075298 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:44.075255    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:44.302701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:44.295659    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:44.322114 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:44.322060    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"]
Oct 30 19:16:44.322378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:44.322322    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6" podUID="2f1c7961-5121-4119-abaa-5df4bdd09220" containerName="istio-proxy" containerID="cri-o://4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322" gracePeriod=30
Oct 30 19:16:44.422593 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:44.422566    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b353a66-fb6e-418d-9538-bf279927ebfc" path="/var/lib/kubelet/pods/0b353a66-fb6e-418d-9538-bf279927ebfc/volumes"
Oct 30 19:16:44.423040 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:44.423026    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cded1de5-2ce0-454c-8426-22558fd26da6" path="/var/lib/kubelet/pods/cded1de5-2ce0-454c-8426-22558fd26da6/volumes"
Oct 30 19:16:44.672178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:44.672138    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:44.848245 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:44.848166    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4" podUID="1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:44.848245 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:44.848178    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz" podUID="9ae0a880-733c-4773-ad10-7239533263df" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:45.071444 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:45.071407    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:45.270814 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:45.270780    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:45.482799 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:45.482759    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:45.845245 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:45.845209    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:46.268708 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:46.268664    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:46.957987 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:46.957950    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:47.674024 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:47.673990    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:48.043959 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.043868    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main" containerID="cri-o://37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82" gracePeriod=45
Oct 30 19:16:48.378592 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.378559    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:48.380398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.380061    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 19:16:48.492087 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.492048    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-home\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-home\") pod \"0ee64038-28e2-4e8e-bb65-2d403a68323c\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") "
Oct 30 19:16:48.492299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.492142    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-tmp-dir\") pod \"0ee64038-28e2-4e8e-bb65-2d403a68323c\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") "
Oct 30 19:16:48.492299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.492183    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-dshm\") pod \"0ee64038-28e2-4e8e-bb65-2d403a68323c\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") "
Oct 30 19:16:48.492299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.492282    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ee64038-28e2-4e8e-bb65-2d403a68323c-kube-api-access-zrr84\" (UniqueName: \"kubernetes.io/projected/0ee64038-28e2-4e8e-bb65-2d403a68323c-kube-api-access-zrr84\") pod \"0ee64038-28e2-4e8e-bb65-2d403a68323c\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") "
Oct 30 19:16:48.492495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.492311    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-model-cache\") pod \"0ee64038-28e2-4e8e-bb65-2d403a68323c\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") "
Oct 30 19:16:48.492495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.492349    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-home" pod "0ee64038-28e2-4e8e-bb65-2d403a68323c" (UID: "0ee64038-28e2-4e8e-bb65-2d403a68323c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:48.492495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.492451    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-tmp-dir" pod "0ee64038-28e2-4e8e-bb65-2d403a68323c" (UID: "0ee64038-28e2-4e8e-bb65-2d403a68323c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:48.492495 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.492462    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0ee64038-28e2-4e8e-bb65-2d403a68323c-tls-certs\" (UniqueName: \"kubernetes.io/secret/0ee64038-28e2-4e8e-bb65-2d403a68323c-tls-certs\") pod \"0ee64038-28e2-4e8e-bb65-2d403a68323c\" (UID: \"0ee64038-28e2-4e8e-bb65-2d403a68323c\") "
Oct 30 19:16:48.492871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.492843    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-model-cache" pod "0ee64038-28e2-4e8e-bb65-2d403a68323c" (UID: "0ee64038-28e2-4e8e-bb65-2d403a68323c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:48.493172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.493076    4925 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-model-cache\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.493342 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.493323    4925 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-home\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.493467 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.493445    4925 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-tmp-dir\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.495595 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.495560    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee64038-28e2-4e8e-bb65-2d403a68323c-kube-api-access-zrr84" pod "0ee64038-28e2-4e8e-bb65-2d403a68323c" (UID: "0ee64038-28e2-4e8e-bb65-2d403a68323c"). InnerVolumeSpecName "kube-api-access-zrr84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:48.495884 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.495846    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-dshm" pod "0ee64038-28e2-4e8e-bb65-2d403a68323c" (UID: "0ee64038-28e2-4e8e-bb65-2d403a68323c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:48.496342 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.496313    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ee64038-28e2-4e8e-bb65-2d403a68323c-tls-certs" pod "0ee64038-28e2-4e8e-bb65-2d403a68323c" (UID: "0ee64038-28e2-4e8e-bb65-2d403a68323c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:48.552998 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.552961    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk" podUID="52fd9b2d-aef2-4a90-872d-5daa0c44dfd3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:48.553470 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.553438    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j" podUID="829ee262-f7f8-495c-ba74-1a6e68f79b2f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:48.594184 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.594111    4925 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ee64038-28e2-4e8e-bb65-2d403a68323c-dshm\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.594184 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.594144    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrr84\" (UniqueName: \"kubernetes.io/projected/0ee64038-28e2-4e8e-bb65-2d403a68323c-kube-api-access-zrr84\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.594184 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.594159    4925 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ee64038-28e2-4e8e-bb65-2d403a68323c-tls-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:48.637213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.634085    4925 generic.go:356] "Generic (PLEG): container finished" podID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerID="37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82" exitCode=0
Oct 30 19:16:48.637213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.634381    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" event={"ID":"0ee64038-28e2-4e8e-bb65-2d403a68323c","Type":"ContainerDied","Data":"37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82"}
Oct 30 19:16:48.637213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.634416    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b" event={"ID":"0ee64038-28e2-4e8e-bb65-2d403a68323c","Type":"ContainerDied","Data":"01df5fb172ca97c285f2a17a3775b1d9987b20bb3535bc23af65f80fc1d20d00"}
Oct 30 19:16:48.637213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.634438    4925 scope.go:122] "RemoveContainer" containerID="37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82"
Oct 30 19:16:48.637213 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.634574    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"
Oct 30 19:16:48.651657 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.651637    4925 scope.go:122] "RemoveContainer" containerID="37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82"
Oct 30 19:16:48.651933 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:48.651914    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82\": container with ID starting with 37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82 not found: ID does not exist" containerID="37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82"
Oct 30 19:16:48.651985 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.651942    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82"} err="failed to get container status \"37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82\": rpc error: code = NotFound desc = could not find container \"37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82\": container with ID starting with 37ad0d3f47308f70db03cae794350a2163cad6886bfafa9825e2c890562f6c82 not found: ID does not exist"
Oct 30 19:16:48.662827 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.662804    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"]
Oct 30 19:16:48.665390 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:48.665369    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-2889dd/distinct-2-w3-2889dd-kserve-cc8747c7b-qk52b"]
Oct 30 19:16:49.138864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.136079    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:49.547942 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.547814    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:49.601581 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.601556    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 19:16:49.645972 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.645942    4925 generic.go:356] "Generic (PLEG): container finished" podID="2f1c7961-5121-4119-abaa-5df4bdd09220" containerID="4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322" exitCode=0
Oct 30 19:16:49.646173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.646002    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"
Oct 30 19:16:49.646173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.646030    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6" event={"ID":"2f1c7961-5121-4119-abaa-5df4bdd09220","Type":"ContainerDied","Data":"4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322"}
Oct 30 19:16:49.646173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.646083    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6" event={"ID":"2f1c7961-5121-4119-abaa-5df4bdd09220","Type":"ContainerDied","Data":"8337a49e392868786a1dee002d579883042ccb7c1a210417dbf0cd47cfb01f4f"}
Oct 30 19:16:49.646173 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.646120    4925 scope.go:122] "RemoveContainer" containerID="4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322"
Oct 30 19:16:49.659007 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.658984    4925 scope.go:122] "RemoveContainer" containerID="4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322"
Oct 30 19:16:49.659338 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:49.659314    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322\": container with ID starting with 4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322 not found: ID does not exist" containerID="4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322"
Oct 30 19:16:49.659418 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.659349    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322"} err="failed to get container status \"4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322\": rpc error: code = NotFound desc = could not find container \"4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322\": container with ID starting with 4317fe3476267672090f022e084cbdd4d60374335f9bcf64dc7ad9471ef37322 not found: ID does not exist"
Oct 30 19:16:49.708926 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.708850    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-podinfo\") pod \"2f1c7961-5121-4119-abaa-5df4bdd09220\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") "
Oct 30 19:16:49.709211 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.709190    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f1c7961-5121-4119-abaa-5df4bdd09220-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f1c7961-5121-4119-abaa-5df4bdd09220-istiod-ca-cert\") pod \"2f1c7961-5121-4119-abaa-5df4bdd09220\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") "
Oct 30 19:16:49.709384 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.709369    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-socket\") pod \"2f1c7961-5121-4119-abaa-5df4bdd09220\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") "
Oct 30 19:16:49.709553 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.709537    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-kube-api-access-psxpx\" (UniqueName: \"kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-kube-api-access-psxpx\") pod \"2f1c7961-5121-4119-abaa-5df4bdd09220\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") "
Oct 30 19:16:49.709703 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.709621    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f1c7961-5121-4119-abaa-5df4bdd09220-istiod-ca-cert" pod "2f1c7961-5121-4119-abaa-5df4bdd09220" (UID: "2f1c7961-5121-4119-abaa-5df4bdd09220"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:49.709770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.709678    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-token\" (UniqueName: \"kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-token\") pod \"2f1c7961-5121-4119-abaa-5df4bdd09220\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") "
Oct 30 19:16:49.709899 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.709878    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-data\") pod \"2f1c7961-5121-4119-abaa-5df4bdd09220\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") "
Oct 30 19:16:49.709959 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.709920    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-certs\") pod \"2f1c7961-5121-4119-abaa-5df4bdd09220\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") "
Oct 30 19:16:49.710022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.709971    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-envoy\") pod \"2f1c7961-5121-4119-abaa-5df4bdd09220\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") "
Oct 30 19:16:49.710022 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.710005    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-credential-socket\") pod \"2f1c7961-5121-4119-abaa-5df4bdd09220\" (UID: \"2f1c7961-5121-4119-abaa-5df4bdd09220\") "
Oct 30 19:16:49.710477 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.710328    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-socket" pod "2f1c7961-5121-4119-abaa-5df4bdd09220" (UID: "2f1c7961-5121-4119-abaa-5df4bdd09220"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:49.710777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.710620    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-certs" pod "2f1c7961-5121-4119-abaa-5df4bdd09220" (UID: "2f1c7961-5121-4119-abaa-5df4bdd09220"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:49.710777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.710684    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f1c7961-5121-4119-abaa-5df4bdd09220-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:49.710777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.710708    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:49.711047 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.711007    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-data" pod "2f1c7961-5121-4119-abaa-5df4bdd09220" (UID: "2f1c7961-5121-4119-abaa-5df4bdd09220"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:49.711590 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.711556    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-credential-socket" pod "2f1c7961-5121-4119-abaa-5df4bdd09220" (UID: "2f1c7961-5121-4119-abaa-5df4bdd09220"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:49.716167 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.716145    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-token" pod "2f1c7961-5121-4119-abaa-5df4bdd09220" (UID: "2f1c7961-5121-4119-abaa-5df4bdd09220"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:49.716434 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.716411    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-podinfo" pod "2f1c7961-5121-4119-abaa-5df4bdd09220" (UID: "2f1c7961-5121-4119-abaa-5df4bdd09220"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:16:49.716523 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.716502    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-envoy" pod "2f1c7961-5121-4119-abaa-5df4bdd09220" (UID: "2f1c7961-5121-4119-abaa-5df4bdd09220"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:49.716932 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.716878    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-kube-api-access-psxpx" pod "2f1c7961-5121-4119-abaa-5df4bdd09220" (UID: "2f1c7961-5121-4119-abaa-5df4bdd09220"). InnerVolumeSpecName "kube-api-access-psxpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:49.811477 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.811438    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psxpx\" (UniqueName: \"kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-kube-api-access-psxpx\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:49.811477 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.811479    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:49.811676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.811490    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:49.811676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.811499    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:49.811676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.811508    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:49.811676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.811516    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f1c7961-5121-4119-abaa-5df4bdd09220-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:49.811676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:49.811525    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f1c7961-5121-4119-abaa-5df4bdd09220-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:50.004739 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:50.004710    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"]
Oct 30 19:16:50.016106 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:50.016066    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-0458df-openshift-default-7c86d79f79-8z4g6"]
Oct 30 19:16:50.079458 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:50.079433    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:50.355851 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:50.355772    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"]
Oct 30 19:16:50.394232 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:50.394199    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"]
Oct 30 19:16:50.427530 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:50.427496    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" path="/var/lib/kubelet/pods/0ee64038-28e2-4e8e-bb65-2d403a68323c/volumes"
Oct 30 19:16:50.429538 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:50.429507    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f1c7961-5121-4119-abaa-5df4bdd09220" path="/var/lib/kubelet/pods/2f1c7961-5121-4119-abaa-5df4bdd09220/volumes"
Oct 30 19:16:50.551933 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:50.551895    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:50.827695 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:50.827661    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:51.035554 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:51.035514    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:51.812579 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:51.812544    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:52.201629 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.201578    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"]
Oct 30 19:16:52.202151 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.202117    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g" podUID="610e800d-7647-40fd-973c-bc58a0214728" containerName="maas-api" containerID="cri-o://3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718" gracePeriod=30
Oct 30 19:16:52.347872 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.347838    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:52.553654 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.553514    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:52.599761 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.599732    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:16:52.675892 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.675849    4925 generic.go:356] "Generic (PLEG): container finished" podID="610e800d-7647-40fd-973c-bc58a0214728" containerID="3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718" exitCode=0
Oct 30 19:16:52.676075 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.675927    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g" event={"ID":"610e800d-7647-40fd-973c-bc58a0214728","Type":"ContainerDied","Data":"3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718"}
Oct 30 19:16:52.676075 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.675966    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g" event={"ID":"610e800d-7647-40fd-973c-bc58a0214728","Type":"ContainerDied","Data":"32f2e3634084c68ff1b5668d1be03fe5ac58e916a6c31ef91bf47c5001044ada"}
Oct 30 19:16:52.676075 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.675990    4925 scope.go:122] "RemoveContainer" containerID="3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718"
Oct 30 19:16:52.676309 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.676127    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"
Oct 30 19:16:52.688970 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.688938    4925 scope.go:122] "RemoveContainer" containerID="3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718"
Oct 30 19:16:52.689360 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:52.689337    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718\": container with ID starting with 3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718 not found: ID does not exist" containerID="3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718"
Oct 30 19:16:52.689457 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.689371    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718"} err="failed to get container status \"3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718\": rpc error: code = NotFound desc = could not find container \"3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718\": container with ID starting with 3fd0a44bfd413e8b8cfe3f32dc533c3cdf5934915c970bd9926a68dcb2983718 not found: ID does not exist"
Oct 30 19:16:52.742724 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.741023    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/610e800d-7647-40fd-973c-bc58a0214728-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/610e800d-7647-40fd-973c-bc58a0214728-openshift-service-ca\") pod \"610e800d-7647-40fd-973c-bc58a0214728\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") "
Oct 30 19:16:52.742724 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.741269    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-metrics-certs\" (UniqueName: \"kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-metrics-certs\") pod \"610e800d-7647-40fd-973c-bc58a0214728\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") "
Oct 30 19:16:52.742724 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.741392    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-maas-api-tls\") pod \"610e800d-7647-40fd-973c-bc58a0214728\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") "
Oct 30 19:16:52.742724 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.742443    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/610e800d-7647-40fd-973c-bc58a0214728-kube-api-access-qt259\" (UniqueName: \"kubernetes.io/projected/610e800d-7647-40fd-973c-bc58a0214728-kube-api-access-qt259\") pod \"610e800d-7647-40fd-973c-bc58a0214728\" (UID: \"610e800d-7647-40fd-973c-bc58a0214728\") "
Oct 30 19:16:52.743582 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.743297    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/610e800d-7647-40fd-973c-bc58a0214728-openshift-service-ca" pod "610e800d-7647-40fd-973c-bc58a0214728" (UID: "610e800d-7647-40fd-973c-bc58a0214728"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:52.769026 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.768835    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/610e800d-7647-40fd-973c-bc58a0214728-kube-api-access-qt259" pod "610e800d-7647-40fd-973c-bc58a0214728" (UID: "610e800d-7647-40fd-973c-bc58a0214728"). InnerVolumeSpecName "kube-api-access-qt259". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:52.769026 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.768939    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-metrics-certs" pod "610e800d-7647-40fd-973c-bc58a0214728" (UID: "610e800d-7647-40fd-973c-bc58a0214728"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:52.773606 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.773568    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-maas-api-tls" pod "610e800d-7647-40fd-973c-bc58a0214728" (UID: "610e800d-7647-40fd-973c-bc58a0214728"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:52.795860 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.795761    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:52.844565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.844492    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qt259\" (UniqueName: \"kubernetes.io/projected/610e800d-7647-40fd-973c-bc58a0214728-kube-api-access-qt259\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:52.844565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.844525    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/610e800d-7647-40fd-973c-bc58a0214728-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:52.844565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.844538    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:52.844565 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:52.844546    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/610e800d-7647-40fd-973c-bc58a0214728-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:53.002621 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:53.002579    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"]
Oct 30 19:16:53.007333 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:53.007300    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f3d6a5-7868cc78b7-wdm5g"]
Oct 30 19:16:53.037121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:53.034546    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:53.250589 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:53.250553    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:53.511263 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:53.511157    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:53.963613 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:53.963566    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:54.423796 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:54.423768    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="610e800d-7647-40fd-973c-bc58a0214728" path="/var/lib/kubelet/pods/610e800d-7647-40fd-973c-bc58a0214728/volumes"
Oct 30 19:16:54.808219 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:54.808130    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:54.847839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:54.847806    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz" podUID="9ae0a880-733c-4773-ad10-7239533263df" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:54.847839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:54.847822    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4" podUID="1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:55.356450 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:55.356411    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv" podUID="57195712-20ec-4a9b-97ce-bfc531287e3a" containerName="payload-processing" containerID="cri-o://07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4" gracePeriod=25
Oct 30 19:16:55.395012 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:55.394944    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw" podUID="f5fb074d-ef9f-44e0-92bd-5e4ab059b78e" containerName="payload-pre-processing" containerID="cri-o://3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63" gracePeriod=25
Oct 30 19:16:55.502773 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:55.502733    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv" podUID="57195712-20ec-4a9b-97ce-bfc531287e3a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:16:55.773133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:55.773070    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:56.086229 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.086199    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"
Oct 30 19:16:56.133510 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.133468    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"
Oct 30 19:16:56.189117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.184032    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-kube-api-access-hcn62\" (UniqueName: \"kubernetes.io/projected/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-kube-api-access-hcn62\") pod \"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3\" (UID: \"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3\") "
Oct 30 19:16:56.189117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.184187    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/829ee262-f7f8-495c-ba74-1a6e68f79b2f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/829ee262-f7f8-495c-ba74-1a6e68f79b2f-plugins-config-volume\") pod \"829ee262-f7f8-495c-ba74-1a6e68f79b2f\" (UID: \"829ee262-f7f8-495c-ba74-1a6e68f79b2f\") "
Oct 30 19:16:56.189117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.184252    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-plugins-config-volume\") pod \"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3\" (UID: \"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3\") "
Oct 30 19:16:56.189117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.184432    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/829ee262-f7f8-495c-ba74-1a6e68f79b2f-kube-api-access-xkcxz\" (UniqueName: \"kubernetes.io/projected/829ee262-f7f8-495c-ba74-1a6e68f79b2f-kube-api-access-xkcxz\") pod \"829ee262-f7f8-495c-ba74-1a6e68f79b2f\" (UID: \"829ee262-f7f8-495c-ba74-1a6e68f79b2f\") "
Oct 30 19:16:56.189117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.185701    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/829ee262-f7f8-495c-ba74-1a6e68f79b2f-plugins-config-volume" pod "829ee262-f7f8-495c-ba74-1a6e68f79b2f" (UID: "829ee262-f7f8-495c-ba74-1a6e68f79b2f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:56.189117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.185940    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-plugins-config-volume" pod "52fd9b2d-aef2-4a90-872d-5daa0c44dfd3" (UID: "52fd9b2d-aef2-4a90-872d-5daa0c44dfd3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:56.189117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.187719    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/829ee262-f7f8-495c-ba74-1a6e68f79b2f-kube-api-access-xkcxz" pod "829ee262-f7f8-495c-ba74-1a6e68f79b2f" (UID: "829ee262-f7f8-495c-ba74-1a6e68f79b2f"). InnerVolumeSpecName "kube-api-access-xkcxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:56.192109 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.190227    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-kube-api-access-hcn62" pod "52fd9b2d-aef2-4a90-872d-5daa0c44dfd3" (UID: "52fd9b2d-aef2-4a90-872d-5daa0c44dfd3"). InnerVolumeSpecName "kube-api-access-hcn62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:56.286898 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.286807    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcn62\" (UniqueName: \"kubernetes.io/projected/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-kube-api-access-hcn62\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:56.286898 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.286842    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/829ee262-f7f8-495c-ba74-1a6e68f79b2f-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:56.286898 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.286856    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:56.286898 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.286871    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkcxz\" (UniqueName: \"kubernetes.io/projected/829ee262-f7f8-495c-ba74-1a6e68f79b2f-kube-api-access-xkcxz\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:56.612049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.611951    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:56.738557 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.738526    4925 generic.go:356] "Generic (PLEG): container finished" podID="52fd9b2d-aef2-4a90-872d-5daa0c44dfd3" containerID="3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9" exitCode=0
Oct 30 19:16:56.738737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.738578    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"
Oct 30 19:16:56.738737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.738617    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk" event={"ID":"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3","Type":"ContainerDied","Data":"3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9"}
Oct 30 19:16:56.738737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.738682    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk" event={"ID":"52fd9b2d-aef2-4a90-872d-5daa0c44dfd3","Type":"ContainerDied","Data":"83c0b3634a1523f474d74b5e90dc08f5ecd77cfda36186863dccea9a91336aed"}
Oct 30 19:16:56.738737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.738717    4925 scope.go:122] "RemoveContainer" containerID="3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9"
Oct 30 19:16:56.740870 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.740844    4925 generic.go:356] "Generic (PLEG): container finished" podID="829ee262-f7f8-495c-ba74-1a6e68f79b2f" containerID="463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2" exitCode=0
Oct 30 19:16:56.741004 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.740884    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j" event={"ID":"829ee262-f7f8-495c-ba74-1a6e68f79b2f","Type":"ContainerDied","Data":"463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2"}
Oct 30 19:16:56.741004 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.740922    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j" event={"ID":"829ee262-f7f8-495c-ba74-1a6e68f79b2f","Type":"ContainerDied","Data":"540789c29b379bea3415def9e451356e4aa1df2f92d9842b5c1940206d5c142f"}
Oct 30 19:16:56.741004 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.740979    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"
Oct 30 19:16:56.749607 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.749571    4925 scope.go:122] "RemoveContainer" containerID="3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9"
Oct 30 19:16:56.749892 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:56.749875    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9\": container with ID starting with 3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9 not found: ID does not exist" containerID="3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9"
Oct 30 19:16:56.749977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.749904    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9"} err="failed to get container status \"3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9\": rpc error: code = NotFound desc = could not find container \"3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9\": container with ID starting with 3366f6a117f7681d6cfbe16fe497fb527a6c1fbf6006809c76b806ace37c37c9 not found: ID does not exist"
Oct 30 19:16:56.749977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.749926    4925 scope.go:122] "RemoveContainer" containerID="463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2"
Oct 30 19:16:56.759998 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.759967    4925 scope.go:122] "RemoveContainer" containerID="463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2"
Oct 30 19:16:56.760432 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:16:56.760408    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2\": container with ID starting with 463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2 not found: ID does not exist" containerID="463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2"
Oct 30 19:16:56.760551 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.760439    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2"} err="failed to get container status \"463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2\": rpc error: code = NotFound desc = could not find container \"463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2\": container with ID starting with 463f5a73d460532dc168ee555eb68e54337d8bb0d59c09628d3f116431f8e9c2 not found: ID does not exist"
Oct 30 19:16:56.767110 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.767073    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"]
Oct 30 19:16:56.772704 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.772684    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-ca2cd5-78bfff4676-xdh5j"]
Oct 30 19:16:56.798065 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.798046    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"]
Oct 30 19:16:56.811374 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:56.811354    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-ca2cd5-5c94cf4c7f-v9vfk"]
Oct 30 19:16:57.372669 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:57.372631    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:57.937108 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:57.937039    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:58.161116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:58.160632    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:58.359467 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:58.359376    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:58.421716 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:58.421682    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52fd9b2d-aef2-4a90-872d-5daa0c44dfd3" path="/var/lib/kubelet/pods/52fd9b2d-aef2-4a90-872d-5daa0c44dfd3/volumes"
Oct 30 19:16:58.422076 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:58.422063    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="829ee262-f7f8-495c-ba74-1a6e68f79b2f" path="/var/lib/kubelet/pods/829ee262-f7f8-495c-ba74-1a6e68f79b2f/volumes"
Oct 30 19:16:58.605201 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:58.605168    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:58.806978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:58.806938    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:59.354144 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:59.354069    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:59.633875 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:59.633778    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"]
Oct 30 19:16:59.634061 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:16:59.634037    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerName="istio-proxy" containerID="cri-o://041d0797ec85b4bc46d103c23097d6da2ef35fd5834ff6b0f32a4a09ac81a751" gracePeriod=30
Oct 30 19:17:00.285969 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:00.285935    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:01.586753 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:01.586709    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:02.280857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:02.280825    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:02.495998 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:02.495958    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:02.536442 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:02.536359    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"]
Oct 30 19:17:02.536637 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:02.536604    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h" podUID="380c3494-b8be-4efd-b495-414a2819d883" containerName="istio-proxy" containerID="cri-o://4740448450e61855e5c0676c85b49eb968b1c05cacfa6499201de2d7215a81ca" gracePeriod=30
Oct 30 19:17:02.543774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:02.543750    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"]
Oct 30 19:17:02.543965 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:02.543942    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerName="istio-proxy" containerID="cri-o://824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c" gracePeriod=30
Oct 30 19:17:02.559818 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:02.559784    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"]
Oct 30 19:17:02.560339 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:02.560305    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerName="istio-proxy" containerID="cri-o://bcf6dff602e924504b405ee69603368a5cecb943ddb22666a0e762646d4a5090" gracePeriod=30
Oct 30 19:17:02.699871 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:02.699792    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:03.238361 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.238292    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:03.730335 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.730306    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"
Oct 30 19:17:03.777948 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.777873    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"
Oct 30 19:17:03.790115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.788432    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ae0a880-733c-4773-ad10-7239533263df-kube-api-access-65flt\" (UniqueName: \"kubernetes.io/projected/9ae0a880-733c-4773-ad10-7239533263df-kube-api-access-65flt\") pod \"9ae0a880-733c-4773-ad10-7239533263df\" (UID: \"9ae0a880-733c-4773-ad10-7239533263df\") "
Oct 30 19:17:03.790115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.788594    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-plugins-config-volume\") pod \"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07\" (UID: \"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07\") "
Oct 30 19:17:03.790115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.788633    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ae0a880-733c-4773-ad10-7239533263df-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ae0a880-733c-4773-ad10-7239533263df-plugins-config-volume\") pod \"9ae0a880-733c-4773-ad10-7239533263df\" (UID: \"9ae0a880-733c-4773-ad10-7239533263df\") "
Oct 30 19:17:03.790115 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.788726    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-kube-api-access-wstcb\" (UniqueName: \"kubernetes.io/projected/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-kube-api-access-wstcb\") pod \"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07\" (UID: \"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07\") "
Oct 30 19:17:03.794118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.792004    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-kube-api-access-wstcb" pod "1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07" (UID: "1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07"). InnerVolumeSpecName "kube-api-access-wstcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:03.798118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.794560    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ae0a880-733c-4773-ad10-7239533263df-kube-api-access-65flt" pod "9ae0a880-733c-4773-ad10-7239533263df" (UID: "9ae0a880-733c-4773-ad10-7239533263df"). InnerVolumeSpecName "kube-api-access-65flt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:03.798118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.794906    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-plugins-config-volume" pod "1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07" (UID: "1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:03.798118 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.796057    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ae0a880-733c-4773-ad10-7239533263df-plugins-config-volume" pod "9ae0a880-733c-4773-ad10-7239533263df" (UID: "9ae0a880-733c-4773-ad10-7239533263df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:03.798709 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.798683    4925 generic.go:356] "Generic (PLEG): container finished" podID="1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07" containerID="3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15" exitCode=0
Oct 30 19:17:03.798914 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.798898    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"
Oct 30 19:17:03.803507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.803163    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4" event={"ID":"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07","Type":"ContainerDied","Data":"3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15"}
Oct 30 19:17:03.803507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.803227    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4" event={"ID":"1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07","Type":"ContainerDied","Data":"77259d9984b9500f31e2b633b333fbe74a3e2edb0a4fb4a45afcdf21d4b42a57"}
Oct 30 19:17:03.803507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.803257    4925 scope.go:122] "RemoveContainer" containerID="3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15"
Oct 30 19:17:03.805018 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.804882    4925 generic.go:356] "Generic (PLEG): container finished" podID="9ae0a880-733c-4773-ad10-7239533263df" containerID="972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f" exitCode=0
Oct 30 19:17:03.805018 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.804954    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz" event={"ID":"9ae0a880-733c-4773-ad10-7239533263df","Type":"ContainerDied","Data":"972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f"}
Oct 30 19:17:03.805018 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.804993    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz" event={"ID":"9ae0a880-733c-4773-ad10-7239533263df","Type":"ContainerDied","Data":"6eea16254cd2daf9f3b90871269ffbef5bd97efdae449291ce20731ebf89552c"}
Oct 30 19:17:03.805261 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.805060    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"
Oct 30 19:17:03.820781 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.820763    4925 scope.go:122] "RemoveContainer" containerID="3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15"
Oct 30 19:17:03.821570 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:17:03.821084    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15\": container with ID starting with 3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15 not found: ID does not exist" containerID="3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15"
Oct 30 19:17:03.821674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.821580    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15"} err="failed to get container status \"3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15\": rpc error: code = NotFound desc = could not find container \"3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15\": container with ID starting with 3a24464d4e940892f51965211fc31064545500ca8862eab679fd9080421acd15 not found: ID does not exist"
Oct 30 19:17:03.821674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.821615    4925 scope.go:122] "RemoveContainer" containerID="972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f"
Oct 30 19:17:03.825196 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.825168    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"]
Oct 30 19:17:03.836318 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.836296    4925 scope.go:122] "RemoveContainer" containerID="972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f"
Oct 30 19:17:03.836631 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:17:03.836603    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f\": container with ID starting with 972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f not found: ID does not exist" containerID="972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f"
Oct 30 19:17:03.836727 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.836641    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f"} err="failed to get container status \"972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f\": rpc error: code = NotFound desc = could not find container \"972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f\": container with ID starting with 972a8bab044d760008343433439847e9832ab71fc7bbc2f5ef4faf462e30640f not found: ID does not exist"
Oct 30 19:17:03.843555 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.843531    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"]
Oct 30 19:17:03.848005 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.847982    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-2889dd-6f44c5c46b-rq9b4"]
Oct 30 19:17:03.865665 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.865640    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"]
Oct 30 19:17:03.869710 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.869682    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2889dd-6cb9cc4d87-95bwz"]
Oct 30 19:17:03.902155 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.902088    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:03.902155 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.902154    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ae0a880-733c-4773-ad10-7239533263df-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:03.902371 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.902172    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wstcb\" (UniqueName: \"kubernetes.io/projected/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07-kube-api-access-wstcb\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:03.902371 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.902185    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65flt\" (UniqueName: \"kubernetes.io/projected/9ae0a880-733c-4773-ad10-7239533263df-kube-api-access-65flt\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:03.988471 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:03.988433    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:04.015090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015058    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"]
Oct 30 19:17:04.015653 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015635    4925 cpu_manager.go:414] "containerMap: removing container" podUID="2f1c7961-5121-4119-abaa-5df4bdd09220" containerName="istio-proxy"
Oct 30 19:17:04.015712 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015655    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f1c7961-5121-4119-abaa-5df4bdd09220" containerName="istio-proxy"
Oct 30 19:17:04.015712 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015664    4925 cpu_manager.go:414] "containerMap: removing container" podUID="50839431-baf0-4a66-a775-54be5ea96723" containerName="cleanup"
Oct 30 19:17:04.015712 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015671    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50839431-baf0-4a66-a775-54be5ea96723" containerName="cleanup"
Oct 30 19:17:04.015712 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015690    4925 cpu_manager.go:414] "containerMap: removing container" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy"
Oct 30 19:17:04.015712 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015698    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy"
Oct 30 19:17:04.015712 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015709    4925 cpu_manager.go:414] "containerMap: removing container" podUID="829ee262-f7f8-495c-ba74-1a6e68f79b2f" containerName="payload-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015716    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="829ee262-f7f8-495c-ba74-1a6e68f79b2f" containerName="payload-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015732    4925 cpu_manager.go:414] "containerMap: removing container" podUID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" containerName="istio-proxy"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015739    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" containerName="istio-proxy"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015754    4925 cpu_manager.go:414] "containerMap: removing container" podUID="3107703b-0cfb-4320-84c9-d023eeea6c36" containerName="payload-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015761    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3107703b-0cfb-4320-84c9-d023eeea6c36" containerName="payload-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015770    4925 cpu_manager.go:414] "containerMap: removing container" podUID="cded1de5-2ce0-454c-8426-22558fd26da6" containerName="payload-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015777    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cded1de5-2ce0-454c-8426-22558fd26da6" containerName="payload-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015787    4925 cpu_manager.go:414] "containerMap: removing container" podUID="ee31e82e-3665-4a8c-b502-429976b05fd9" containerName="maas-api"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015793    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee31e82e-3665-4a8c-b502-429976b05fd9" containerName="maas-api"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015803    4925 cpu_manager.go:414] "containerMap: removing container" podUID="0b353a66-fb6e-418d-9538-bf279927ebfc" containerName="payload-pre-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015807    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b353a66-fb6e-418d-9538-bf279927ebfc" containerName="payload-pre-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015813    4925 cpu_manager.go:414] "containerMap: removing container" podUID="2640e252-a00f-462c-bd40-760ba7bc79e0" containerName="payload-pre-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015818    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2640e252-a00f-462c-bd40-760ba7bc79e0" containerName="payload-pre-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015825    4925 cpu_manager.go:414] "containerMap: removing container" podUID="82c8263f-5c5f-4cce-adcb-8e15e22a8158" containerName="payload-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015831    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82c8263f-5c5f-4cce-adcb-8e15e22a8158" containerName="payload-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015836    4925 cpu_manager.go:414] "containerMap: removing container" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015841    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015847    4925 cpu_manager.go:414] "containerMap: removing container" podUID="129fe17f-da6a-40e0-8b8f-1c4764b5bf4c" containerName="payload-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015855    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="129fe17f-da6a-40e0-8b8f-1c4764b5bf4c" containerName="payload-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015864    4925 cpu_manager.go:414] "containerMap: removing container" podUID="93cec944-2c6d-4e13-8fad-c663bd7934ae" containerName="maas-api"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015871    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93cec944-2c6d-4e13-8fad-c663bd7934ae" containerName="maas-api"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015879    4925 cpu_manager.go:414] "containerMap: removing container" podUID="11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964" containerName="revoke-keys"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015886    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964" containerName="revoke-keys"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015899    4925 cpu_manager.go:414] "containerMap: removing container" podUID="af4ba1e6-4f4c-42c6-bc34-4eafcc18e428" containerName="payload-pre-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015904    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af4ba1e6-4f4c-42c6-bc34-4eafcc18e428" containerName="payload-pre-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015910    4925 cpu_manager.go:414] "containerMap: removing container" podUID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" containerName="cleanup"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015914    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" containerName="cleanup"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015918    4925 cpu_manager.go:414] "containerMap: removing container" podUID="50839431-baf0-4a66-a775-54be5ea96723" containerName="cleanup"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015922    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50839431-baf0-4a66-a775-54be5ea96723" containerName="cleanup"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015930    4925 cpu_manager.go:414] "containerMap: removing container" podUID="c83546ba-7aa3-4999-a888-f317b440593b" containerName="payload-processing"
Oct 30 19:17:04.015927 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015937    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c83546ba-7aa3-4999-a888-f317b440593b" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015951    4925 cpu_manager.go:414] "containerMap: removing container" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015956    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015967    4925 cpu_manager.go:414] "containerMap: removing container" podUID="52fd9b2d-aef2-4a90-872d-5daa0c44dfd3" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015971    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52fd9b2d-aef2-4a90-872d-5daa0c44dfd3" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015979    4925 cpu_manager.go:414] "containerMap: removing container" podUID="3c79b524-09c4-4e29-8a1c-2306621221db" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015983    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c79b524-09c4-4e29-8a1c-2306621221db" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015989    4925 cpu_manager.go:414] "containerMap: removing container" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.015993    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016002    4925 cpu_manager.go:414] "containerMap: removing container" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016006    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016014    4925 cpu_manager.go:414] "containerMap: removing container" podUID="8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016022    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016031    4925 cpu_manager.go:414] "containerMap: removing container" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016035    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016051    4925 cpu_manager.go:414] "containerMap: removing container" podUID="a98e91b2-25b6-4853-832f-e912c92ea068" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016056    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a98e91b2-25b6-4853-832f-e912c92ea068" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016065    4925 cpu_manager.go:414] "containerMap: removing container" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016069    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016076    4925 cpu_manager.go:414] "containerMap: removing container" podUID="7495850d-1b3d-4df3-8424-ebe372d9eb88" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016080    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7495850d-1b3d-4df3-8424-ebe372d9eb88" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016087    4925 cpu_manager.go:414] "containerMap: removing container" podUID="547ccaea-7848-46dd-a638-3411f31835f7" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016112    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="547ccaea-7848-46dd-a638-3411f31835f7" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016121    4925 cpu_manager.go:414] "containerMap: removing container" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016125    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016131    4925 cpu_manager.go:414] "containerMap: removing container" podUID="9ae0a880-733c-4773-ad10-7239533263df" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016136    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ae0a880-733c-4773-ad10-7239533263df" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016141    4925 cpu_manager.go:414] "containerMap: removing container" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016147    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016158    4925 cpu_manager.go:414] "containerMap: removing container" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016162    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016169    4925 cpu_manager.go:414] "containerMap: removing container" podUID="c71c1a5f-6d34-4b5b-854c-ae1577419337" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016175    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c71c1a5f-6d34-4b5b-854c-ae1577419337" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016185    4925 cpu_manager.go:414] "containerMap: removing container" podUID="3b55892d-8b94-4401-bc08-65bd855471f5" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016192    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b55892d-8b94-4401-bc08-65bd855471f5" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016199    4925 cpu_manager.go:414] "containerMap: removing container" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016203    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016209    4925 cpu_manager.go:414] "containerMap: removing container" podUID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016213    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016221    4925 cpu_manager.go:414] "containerMap: removing container" podUID="dfa60598-c22f-4caa-a3dc-e3092197742f" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016225    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfa60598-c22f-4caa-a3dc-e3092197742f" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016231    4925 cpu_manager.go:414] "containerMap: removing container" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016236    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016246    4925 cpu_manager.go:414] "containerMap: removing container" podUID="6c157a5a-f699-4ea7-a5d7-7b4bb851ef88" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016251    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c157a5a-f699-4ea7-a5d7-7b4bb851ef88" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016264    4925 cpu_manager.go:414] "containerMap: removing container" podUID="86f2d7f0-ab22-4a23-81c7-63536b3fb2ad" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016270    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86f2d7f0-ab22-4a23-81c7-63536b3fb2ad" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016275    4925 cpu_manager.go:414] "containerMap: removing container" podUID="0fbcbf42-c4b7-4531-a941-2be9622bd9d5" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016279    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0fbcbf42-c4b7-4531-a941-2be9622bd9d5" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016286    4925 cpu_manager.go:414] "containerMap: removing container" podUID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016290    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016295    4925 cpu_manager.go:414] "containerMap: removing container" podUID="610e800d-7647-40fd-973c-bc58a0214728" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016299    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="610e800d-7647-40fd-973c-bc58a0214728" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016306    4925 cpu_manager.go:414] "containerMap: removing container" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016310    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016318    4925 cpu_manager.go:414] "containerMap: removing container" podUID="cfe33049-c39b-45f7-97c2-78dbd003c401" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016326    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfe33049-c39b-45f7-97c2-78dbd003c401" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016341    4925 cpu_manager.go:414] "containerMap: removing container" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016347    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016357    4925 cpu_manager.go:414] "containerMap: removing container" podUID="bfd0603c-3ee0-476c-916c-e9c4f49b1077" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016364    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bfd0603c-3ee0-476c-916c-e9c4f49b1077" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016374    4925 cpu_manager.go:414] "containerMap: removing container" podUID="e4469a92-c400-468a-8309-13f96c4509dc" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016378    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4469a92-c400-468a-8309-13f96c4509dc" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016384    4925 cpu_manager.go:414] "containerMap: removing container" podUID="f531c729-0f21-45d5-b635-e8613eda326b" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016388    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f531c729-0f21-45d5-b635-e8613eda326b" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016396    4925 cpu_manager.go:414] "containerMap: removing container" podUID="a4d3263e-e6be-4c1e-9700-b654cedc6258" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016401    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4d3263e-e6be-4c1e-9700-b654cedc6258" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016408    4925 cpu_manager.go:414] "containerMap: removing container" podUID="0a2211bc-9633-49ee-b65e-7c39f753357d" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016413    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a2211bc-9633-49ee-b65e-7c39f753357d" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016419    4925 cpu_manager.go:414] "containerMap: removing container" podUID="1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016425    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016435    4925 cpu_manager.go:414] "containerMap: removing container" podUID="2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016441    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016553    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="93cec944-2c6d-4e13-8fad-c663bd7934ae" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016561    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016569    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="11d3dcae-cfe9-4e5e-b9c2-0e2b67f30964" containerName="revoke-keys"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016576    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="af4ba1e6-4f4c-42c6-bc34-4eafcc18e428" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016588    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016596    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="50839431-baf0-4a66-a775-54be5ea96723" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016602    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="c83546ba-7aa3-4999-a888-f317b440593b" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016621    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016629    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="52fd9b2d-aef2-4a90-872d-5daa0c44dfd3" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016637    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c79b524-09c4-4e29-8a1c-2306621221db" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016642    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e69194e-043b-48dc-aa47-4f6d378a09a4" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016650    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ee64038-28e2-4e8e-bb65-2d403a68323c" containerName="main"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016654    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e6a4811-2f5c-4fc8-9d66-e85a44e7dc76" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016661    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016668    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="a98e91b2-25b6-4853-832f-e912c92ea068" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016677    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="97223b58-c28a-4e9b-b84c-dbd10acb606e" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016686    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="7495850d-1b3d-4df3-8424-ebe372d9eb88" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016692    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="547ccaea-7848-46dd-a638-3411f31835f7" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016697    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae0e1c00-9ab0-4e0b-911b-31e1843d9232" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016703    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ae0a880-733c-4773-ad10-7239533263df" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016708    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016714    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="82e8999f-b612-48bf-a4ec-c03fb7597b04" containerName="main"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016719    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="c71c1a5f-6d34-4b5b-854c-ae1577419337" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016724    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b55892d-8b94-4401-bc08-65bd855471f5" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016733    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="673b9cea-e53a-46be-968f-3ad97d7deb2d" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016739    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfa60598-c22f-4caa-a3dc-e3092197742f" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016744    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d4e5708-1d38-4db4-baef-f5863d72c2d1" containerName="main"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016750    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c157a5a-f699-4ea7-a5d7-7b4bb851ef88" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016757    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="86f2d7f0-ab22-4a23-81c7-63536b3fb2ad" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016762    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="0fbcbf42-c4b7-4531-a941-2be9622bd9d5" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016768    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016773    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="610e800d-7647-40fd-973c-bc58a0214728" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016777    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016782    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfe33049-c39b-45f7-97c2-78dbd003c401" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016788    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7b6c4e3-a775-4e0f-abe5-a5051ee2c2f8" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016793    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="bfd0603c-3ee0-476c-916c-e9c4f49b1077" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016799    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4469a92-c400-468a-8309-13f96c4509dc" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016803    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="f531c729-0f21-45d5-b635-e8613eda326b" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016809    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4d3263e-e6be-4c1e-9700-b654cedc6258" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016814    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a2211bc-9633-49ee-b65e-7c39f753357d" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016820    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016825    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="50839431-baf0-4a66-a775-54be5ea96723" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016831    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d4a8452-bc2e-4f5a-80bc-8a971a4a05ab" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016836    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f1c7961-5121-4119-abaa-5df4bdd09220" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016841    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="50839431-baf0-4a66-a775-54be5ea96723" containerName="cleanup"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016846    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="829ee262-f7f8-495c-ba74-1a6e68f79b2f" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016852    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="532cf3a3-18af-48ac-8df2-ce52f16f98ea" containerName="istio-proxy"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016857    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="3107703b-0cfb-4320-84c9-d023eeea6c36" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016863    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="cded1de5-2ce0-454c-8426-22558fd26da6" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016869    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee31e82e-3665-4a8c-b502-429976b05fd9" containerName="maas-api"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016875    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b353a66-fb6e-418d-9538-bf279927ebfc" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016880    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="2640e252-a00f-462c-bd40-760ba7bc79e0" containerName="payload-pre-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016886    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="82c8263f-5c5f-4cce-adcb-8e15e22a8158" containerName="payload-processing"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016891    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="3958fcc7-db32-4565-a937-8fccb3bf644e" containerName="main"
Oct 30 19:17:04.017398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.016895    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="129fe17f-da6a-40e0-8b8f-1c4764b5bf4c" containerName="payload-processing"
Oct 30 19:17:04.022269 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.021766    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:04.027984 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.027928    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:17:04.028164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.028036    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-94hcn\""
Oct 30 19:17:04.038043 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.038018    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"]
Oct 30 19:17:04.123120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.122829    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7f6346b4-9wf45\" (UID: \"d2eed241-cb88-48e4-9cba-38e63d153cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:04.123120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.122915    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-552qg\" (UniqueName: \"kubernetes.io/projected/d2eed241-cb88-48e4-9cba-38e63d153cc3-kube-api-access-552qg\") pod \"maas-api-revoke-sub-7f6346b4-9wf45\" (UID: \"d2eed241-cb88-48e4-9cba-38e63d153cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:04.123120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.122980    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7f6346b4-9wf45\" (UID: \"d2eed241-cb88-48e4-9cba-38e63d153cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:04.188156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.187241    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:04.224058 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.224029    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7f6346b4-9wf45\" (UID: \"d2eed241-cb88-48e4-9cba-38e63d153cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:04.224283 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.224076    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-552qg\" (UniqueName: \"kubernetes.io/projected/d2eed241-cb88-48e4-9cba-38e63d153cc3-kube-api-access-552qg\") pod \"maas-api-revoke-sub-7f6346b4-9wf45\" (UID: \"d2eed241-cb88-48e4-9cba-38e63d153cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:04.224283 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.224129    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7f6346b4-9wf45\" (UID: \"d2eed241-cb88-48e4-9cba-38e63d153cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:04.224758 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.224731    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7f6346b4-9wf45\" (UID: \"d2eed241-cb88-48e4-9cba-38e63d153cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:04.226690 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.226667    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7f6346b4-9wf45\" (UID: \"d2eed241-cb88-48e4-9cba-38e63d153cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:04.233590 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.233561    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-552qg\" (UniqueName: \"kubernetes.io/projected/d2eed241-cb88-48e4-9cba-38e63d153cc3-kube-api-access-552qg\") pod \"maas-api-revoke-sub-7f6346b4-9wf45\" (UID: \"d2eed241-cb88-48e4-9cba-38e63d153cc3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:04.336057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.335991    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:04.433085 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.433048    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07" path="/var/lib/kubelet/pods/1670ef9a-3a4d-43ca-b0fd-4ca6e6735b07/volumes"
Oct 30 19:17:04.433839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.433812    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ae0a880-733c-4773-ad10-7239533263df" path="/var/lib/kubelet/pods/9ae0a880-733c-4773-ad10-7239533263df/volumes"
Oct 30 19:17:04.490763 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.490482    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw" podUID="f5fb074d-ef9f-44e0-92bd-5e4ab059b78e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:17:04.537492 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.537456    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:04.631034 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.631008    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"]
Oct 30 19:17:04.633485 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:17:04.633453    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2eed241_cb88_48e4_9cba_38e63d153cc3.slice/crio-d4dfb0fc2e17c1413bbcb4791b07cec5e06ecd9ee8656cb80cc5a38dbb08c0f8 WatchSource:0}: Error finding container d4dfb0fc2e17c1413bbcb4791b07cec5e06ecd9ee8656cb80cc5a38dbb08c0f8: Status 404 returned error can't find the container with id d4dfb0fc2e17c1413bbcb4791b07cec5e06ecd9ee8656cb80cc5a38dbb08c0f8
Oct 30 19:17:04.816157 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.816126    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45" event={"ID":"d2eed241-cb88-48e4-9cba-38e63d153cc3","Type":"ContainerStarted","Data":"d4dfb0fc2e17c1413bbcb4791b07cec5e06ecd9ee8656cb80cc5a38dbb08c0f8"}
Oct 30 19:17:04.819637 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.818671    4925 generic.go:356] "Generic (PLEG): container finished" podID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerID="041d0797ec85b4bc46d103c23097d6da2ef35fd5834ff6b0f32a4a09ac81a751" exitCode=0
Oct 30 19:17:04.819637 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.818743    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" event={"ID":"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5","Type":"ContainerDied","Data":"041d0797ec85b4bc46d103c23097d6da2ef35fd5834ff6b0f32a4a09ac81a751"}
Oct 30 19:17:04.913645 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:04.913621    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 19:17:05.038372 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038342    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-socket\") pod \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") "
Oct 30 19:17:05.038540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038385    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-data\") pod \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") "
Oct 30 19:17:05.038540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038429    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-kube-api-access-82fwn\" (UniqueName: \"kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-kube-api-access-82fwn\") pod \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") "
Oct 30 19:17:05.038540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038490    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-envoy\") pod \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") "
Oct 30 19:17:05.038540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038514    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-credential-socket\") pod \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") "
Oct 30 19:17:05.038774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038617    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istiod-ca-cert\") pod \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") "
Oct 30 19:17:05.038774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038657    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-token\" (UniqueName: \"kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-token\") pod \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") "
Oct 30 19:17:05.038774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038653    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-socket" pod "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" (UID: "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:05.038774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038705    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-certs\") pod \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") "
Oct 30 19:17:05.038774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038702    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-data" pod "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" (UID: "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:05.038774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038740    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-podinfo\") pod \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\" (UID: \"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5\") "
Oct 30 19:17:05.039146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038907    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-certs" pod "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" (UID: "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:05.039146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.038988    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istiod-ca-cert" pod "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" (UID: "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:05.039146 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.039070    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-credential-socket" pod "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" (UID: "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:05.039494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.039257    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:05.039494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.039274    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:05.039494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.039284    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:05.039494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.039293    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:05.039494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.039301    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:05.040824 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.040803    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-kube-api-access-82fwn" pod "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" (UID: "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5"). InnerVolumeSpecName "kube-api-access-82fwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:05.041109 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.041079    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-podinfo" pod "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" (UID: "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:05.041487 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.041466    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-envoy" pod "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" (UID: "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:05.041563 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.041506    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-token" pod "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" (UID: "2f91ff3f-7f4b-413b-a657-df0a3e19d1d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:05.140013 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.139978    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:05.140013 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.140010    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:05.140231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.140021    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82fwn\" (UniqueName: \"kubernetes.io/projected/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-kube-api-access-82fwn\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:05.140231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.140032    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:05.451719 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.450448    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:05.502485 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.502449    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv" podUID="57195712-20ec-4a9b-97ce-bfc531287e3a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:17:05.826577 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.826499    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"
Oct 30 19:17:05.826577 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.826510    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4" event={"ID":"2f91ff3f-7f4b-413b-a657-df0a3e19d1d5","Type":"ContainerDied","Data":"cfd13ec0bc4473595cdf30602fef0c2bc89e325e709924671839b9d320fa9bdf"}
Oct 30 19:17:05.827080 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.826586    4925 scope.go:122] "RemoveContainer" containerID="041d0797ec85b4bc46d103c23097d6da2ef35fd5834ff6b0f32a4a09ac81a751"
Oct 30 19:17:05.831392 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.831360    4925 generic.go:356] "Generic (PLEG): container finished" podID="d2eed241-cb88-48e4-9cba-38e63d153cc3" containerID="a74b4cd7203c2aef908bf1edb996ea42ee6acc9a86220cc6d6d6b531d25550da" exitCode=0
Oct 30 19:17:05.831529 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.831402    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45" event={"ID":"d2eed241-cb88-48e4-9cba-38e63d153cc3","Type":"ContainerDied","Data":"a74b4cd7203c2aef908bf1edb996ea42ee6acc9a86220cc6d6d6b531d25550da"}
Oct 30 19:17:05.868581 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.868551    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"]
Oct 30 19:17:05.875838 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:05.875810    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-68cb9b4b95-k9sm4"]
Oct 30 19:17:06.102696 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:06.102617    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:06.313695 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:06.313660    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"]
Oct 30 19:17:06.313932 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:06.313887    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy" containerID="cri-o://8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84" gracePeriod=30
Oct 30 19:17:06.421816 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:06.421776    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" path="/var/lib/kubelet/pods/2f91ff3f-7f4b-413b-a657-df0a3e19d1d5/volumes"
Oct 30 19:17:06.468123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:06.455629    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:06.661374 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:06.661341    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:06.991199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:06.991175    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:07.067396 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.067367    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-openshift-ca\") pod \"d2eed241-cb88-48e4-9cba-38e63d153cc3\" (UID: \"d2eed241-cb88-48e4-9cba-38e63d153cc3\") "
Oct 30 19:17:07.067574 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.067491    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-xks-ca\") pod \"d2eed241-cb88-48e4-9cba-38e63d153cc3\" (UID: \"d2eed241-cb88-48e4-9cba-38e63d153cc3\") "
Oct 30 19:17:07.067646 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.067596    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2eed241-cb88-48e4-9cba-38e63d153cc3-kube-api-access-552qg\" (UniqueName: \"kubernetes.io/projected/d2eed241-cb88-48e4-9cba-38e63d153cc3-kube-api-access-552qg\") pod \"d2eed241-cb88-48e4-9cba-38e63d153cc3\" (UID: \"d2eed241-cb88-48e4-9cba-38e63d153cc3\") "
Oct 30 19:17:07.067718 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.067687    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-openshift-ca" pod "d2eed241-cb88-48e4-9cba-38e63d153cc3" (UID: "d2eed241-cb88-48e4-9cba-38e63d153cc3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:07.067990 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.067969    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-openshift-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.069756 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.069732    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-xks-ca" pod "d2eed241-cb88-48e4-9cba-38e63d153cc3" (UID: "d2eed241-cb88-48e4-9cba-38e63d153cc3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:07.069874 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.069758    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2eed241-cb88-48e4-9cba-38e63d153cc3-kube-api-access-552qg" pod "d2eed241-cb88-48e4-9cba-38e63d153cc3" (UID: "d2eed241-cb88-48e4-9cba-38e63d153cc3"). InnerVolumeSpecName "kube-api-access-552qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:07.168879 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.168854    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d2eed241-cb88-48e4-9cba-38e63d153cc3-maas-api-xks-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.168879 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.168879    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-552qg\" (UniqueName: \"kubernetes.io/projected/d2eed241-cb88-48e4-9cba-38e63d153cc3-kube-api-access-552qg\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.477924 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.477828    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:07.832546 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.832520    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:17:07.858514 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.856960    4925 generic.go:356] "Generic (PLEG): container finished" podID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerID="824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c" exitCode=0
Oct 30 19:17:07.858514 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.857059    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" event={"ID":"227555e2-ac4e-4033-a23b-0c43c6eb37ac","Type":"ContainerDied","Data":"824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c"}
Oct 30 19:17:07.858514 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.857137    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7" event={"ID":"227555e2-ac4e-4033-a23b-0c43c6eb37ac","Type":"ContainerDied","Data":"4c1b2a693b1896bf4b996c1121570ef7aa6ddfd1f8a0f1774010affa30c2944d"}
Oct 30 19:17:07.858514 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.857172    4925 scope.go:122] "RemoveContainer" containerID="824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c"
Oct 30 19:17:07.861546 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.861461    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"
Oct 30 19:17:07.882240 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.869546    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45" event={"ID":"d2eed241-cb88-48e4-9cba-38e63d153cc3","Type":"ContainerDied","Data":"d4dfb0fc2e17c1413bbcb4791b07cec5e06ecd9ee8656cb80cc5a38dbb08c0f8"}
Oct 30 19:17:07.882240 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.869595    4925 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4dfb0fc2e17c1413bbcb4791b07cec5e06ecd9ee8656cb80cc5a38dbb08c0f8"
Oct 30 19:17:07.882240 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.869657    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7f6346b4-9wf45"
Oct 30 19:17:07.883529 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.883491    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-envoy\") pod \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") "
Oct 30 19:17:07.884007 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.883985    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istiod-ca-cert\") pod \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") "
Oct 30 19:17:07.884133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.884026    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-credential-socket\") pod \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") "
Oct 30 19:17:07.884133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.884109    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-socket\") pod \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") "
Oct 30 19:17:07.884275 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.884261    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-data\") pod \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") "
Oct 30 19:17:07.884344 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.884292    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-token\" (UniqueName: \"kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-token\") pod \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") "
Oct 30 19:17:07.884413 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.884396    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-certs\") pod \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") "
Oct 30 19:17:07.884470 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.884444    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-podinfo\") pod \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") "
Oct 30 19:17:07.884536 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.884508    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-kube-api-access-7gwj6\" (UniqueName: \"kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-kube-api-access-7gwj6\") pod \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\" (UID: \"227555e2-ac4e-4033-a23b-0c43c6eb37ac\") "
Oct 30 19:17:07.885033 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.885002    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-data" pod "227555e2-ac4e-4033-a23b-0c43c6eb37ac" (UID: "227555e2-ac4e-4033-a23b-0c43c6eb37ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.885262 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.885207    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-socket" pod "227555e2-ac4e-4033-a23b-0c43c6eb37ac" (UID: "227555e2-ac4e-4033-a23b-0c43c6eb37ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.885507 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.885485    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-certs" pod "227555e2-ac4e-4033-a23b-0c43c6eb37ac" (UID: "227555e2-ac4e-4033-a23b-0c43c6eb37ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.885693 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.885674    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istiod-ca-cert" pod "227555e2-ac4e-4033-a23b-0c43c6eb37ac" (UID: "227555e2-ac4e-4033-a23b-0c43c6eb37ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:07.886400 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.886358    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-envoy" pod "227555e2-ac4e-4033-a23b-0c43c6eb37ac" (UID: "227555e2-ac4e-4033-a23b-0c43c6eb37ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.894848 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.891464    4925 generic.go:356] "Generic (PLEG): container finished" podID="380c3494-b8be-4efd-b495-414a2819d883" containerID="4740448450e61855e5c0676c85b49eb968b1c05cacfa6499201de2d7215a81ca" exitCode=0
Oct 30 19:17:07.894848 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.891595    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h" event={"ID":"380c3494-b8be-4efd-b495-414a2819d883","Type":"ContainerDied","Data":"4740448450e61855e5c0676c85b49eb968b1c05cacfa6499201de2d7215a81ca"}
Oct 30 19:17:07.894848 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.894461    4925 generic.go:356] "Generic (PLEG): container finished" podID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerID="bcf6dff602e924504b405ee69603368a5cecb943ddb22666a0e762646d4a5090" exitCode=0
Oct 30 19:17:07.894848 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.894649    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" event={"ID":"67e4a918-26ba-4ebd-8822-a182e8c84418","Type":"ContainerDied","Data":"bcf6dff602e924504b405ee69603368a5cecb943ddb22666a0e762646d4a5090"}
Oct 30 19:17:07.900650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.891034    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-token" pod "227555e2-ac4e-4033-a23b-0c43c6eb37ac" (UID: "227555e2-ac4e-4033-a23b-0c43c6eb37ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:07.904273 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.904243    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-kube-api-access-7gwj6" pod "227555e2-ac4e-4033-a23b-0c43c6eb37ac" (UID: "227555e2-ac4e-4033-a23b-0c43c6eb37ac"). InnerVolumeSpecName "kube-api-access-7gwj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:07.909711 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.908603    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-podinfo" pod "227555e2-ac4e-4033-a23b-0c43c6eb37ac" (UID: "227555e2-ac4e-4033-a23b-0c43c6eb37ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:07.910826 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.910675    4925 scope.go:122] "RemoveContainer" containerID="824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c"
Oct 30 19:17:07.911755 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.911208    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-credential-socket" pod "227555e2-ac4e-4033-a23b-0c43c6eb37ac" (UID: "227555e2-ac4e-4033-a23b-0c43c6eb37ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.911755 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:17:07.911720    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c\": container with ID starting with 824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c not found: ID does not exist" containerID="824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c"
Oct 30 19:17:07.911922 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.911758    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c"} err="failed to get container status \"824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c\": rpc error: code = NotFound desc = could not find container \"824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c\": container with ID starting with 824486f13d499e8a0893275275cc4b77f135cb71e422de27e7cc1251aa7e3c7c not found: ID does not exist"
Oct 30 19:17:07.916624 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.916593    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:17:07.921411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.921394    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:17:07.985664 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.985629    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-token\" (UniqueName: \"kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-token\") pod \"67e4a918-26ba-4ebd-8822-a182e8c84418\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") "
Oct 30 19:17:07.985804 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.985697    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-credential-socket\") pod \"380c3494-b8be-4efd-b495-414a2819d883\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") "
Oct 30 19:17:07.985804 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.985744    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-data\") pod \"67e4a918-26ba-4ebd-8822-a182e8c84418\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") "
Oct 30 19:17:07.985804 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.985775    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-credential-socket\") pod \"67e4a918-26ba-4ebd-8822-a182e8c84418\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") "
Oct 30 19:17:07.986000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.985810    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-kube-api-access-qf268\" (UniqueName: \"kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-kube-api-access-qf268\") pod \"67e4a918-26ba-4ebd-8822-a182e8c84418\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") "
Oct 30 19:17:07.986000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.985844    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/380c3494-b8be-4efd-b495-414a2819d883-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/380c3494-b8be-4efd-b495-414a2819d883-istio-podinfo\") pod \"380c3494-b8be-4efd-b495-414a2819d883\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") "
Oct 30 19:17:07.986000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.985878    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-socket\") pod \"67e4a918-26ba-4ebd-8822-a182e8c84418\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") "
Oct 30 19:17:07.986000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.985945    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-envoy\") pod \"380c3494-b8be-4efd-b495-414a2819d883\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") "
Oct 30 19:17:07.986000 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.985973    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-istio-token\" (UniqueName: \"kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-istio-token\") pod \"380c3494-b8be-4efd-b495-414a2819d883\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") "
Oct 30 19:17:07.986303 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.986132    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-envoy\") pod \"67e4a918-26ba-4ebd-8822-a182e8c84418\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") "
Oct 30 19:17:07.986303 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.986178    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-kube-api-access-w4hr5\" (UniqueName: \"kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-kube-api-access-w4hr5\") pod \"380c3494-b8be-4efd-b495-414a2819d883\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") "
Oct 30 19:17:07.986303 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.986211    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-certs\") pod \"380c3494-b8be-4efd-b495-414a2819d883\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") "
Oct 30 19:17:07.986303 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.986247    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/380c3494-b8be-4efd-b495-414a2819d883-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/380c3494-b8be-4efd-b495-414a2819d883-istiod-ca-cert\") pod \"380c3494-b8be-4efd-b495-414a2819d883\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") "
Oct 30 19:17:07.986518 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.986307    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-certs\") pod \"67e4a918-26ba-4ebd-8822-a182e8c84418\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") "
Oct 30 19:17:07.986518 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.986345    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/67e4a918-26ba-4ebd-8822-a182e8c84418-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e4a918-26ba-4ebd-8822-a182e8c84418-istiod-ca-cert\") pod \"67e4a918-26ba-4ebd-8822-a182e8c84418\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") "
Oct 30 19:17:07.986518 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.986378    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-socket\") pod \"380c3494-b8be-4efd-b495-414a2819d883\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") "
Oct 30 19:17:07.986518 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.986426    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-data\") pod \"380c3494-b8be-4efd-b495-414a2819d883\" (UID: \"380c3494-b8be-4efd-b495-414a2819d883\") "
Oct 30 19:17:07.986518 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.986504    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-podinfo\") pod \"67e4a918-26ba-4ebd-8822-a182e8c84418\" (UID: \"67e4a918-26ba-4ebd-8822-a182e8c84418\") "
Oct 30 19:17:07.987257 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.987223    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-credential-socket" pod "67e4a918-26ba-4ebd-8822-a182e8c84418" (UID: "67e4a918-26ba-4ebd-8822-a182e8c84418"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.987567 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.987544    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.987674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.987573    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.987674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.987590    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.987674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.987605    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.987674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.987620    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gwj6\" (UniqueName: \"kubernetes.io/projected/227555e2-ac4e-4033-a23b-0c43c6eb37ac-kube-api-access-7gwj6\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.987674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.987633    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.987674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.987647    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.987674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.987660    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/227555e2-ac4e-4033-a23b-0c43c6eb37ac-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.987674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.987675    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.988137 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.987689    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/227555e2-ac4e-4033-a23b-0c43c6eb37ac-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.988267 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.988226    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-credential-socket" pod "380c3494-b8be-4efd-b495-414a2819d883" (UID: "380c3494-b8be-4efd-b495-414a2819d883"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.988604 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.988260    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-certs" pod "67e4a918-26ba-4ebd-8822-a182e8c84418" (UID: "67e4a918-26ba-4ebd-8822-a182e8c84418"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.988718 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.988698    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67e4a918-26ba-4ebd-8822-a182e8c84418-istiod-ca-cert" pod "67e4a918-26ba-4ebd-8822-a182e8c84418" (UID: "67e4a918-26ba-4ebd-8822-a182e8c84418"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:07.989208 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.988934    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-socket" pod "380c3494-b8be-4efd-b495-414a2819d883" (UID: "380c3494-b8be-4efd-b495-414a2819d883"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.989208 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.988976    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-data" pod "380c3494-b8be-4efd-b495-414a2819d883" (UID: "380c3494-b8be-4efd-b495-414a2819d883"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.989475 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.989445    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-certs" pod "380c3494-b8be-4efd-b495-414a2819d883" (UID: "380c3494-b8be-4efd-b495-414a2819d883"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.989658 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.989625    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-data" pod "67e4a918-26ba-4ebd-8822-a182e8c84418" (UID: "67e4a918-26ba-4ebd-8822-a182e8c84418"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.989821 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.989797    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-socket" pod "67e4a918-26ba-4ebd-8822-a182e8c84418" (UID: "67e4a918-26ba-4ebd-8822-a182e8c84418"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.990120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.990081    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/380c3494-b8be-4efd-b495-414a2819d883-istiod-ca-cert" pod "380c3494-b8be-4efd-b495-414a2819d883" (UID: "380c3494-b8be-4efd-b495-414a2819d883"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:07.990459 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.990343    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-envoy" pod "67e4a918-26ba-4ebd-8822-a182e8c84418" (UID: "67e4a918-26ba-4ebd-8822-a182e8c84418"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.992395 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.992360    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-envoy" pod "380c3494-b8be-4efd-b495-414a2819d883" (UID: "380c3494-b8be-4efd-b495-414a2819d883"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.993412 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.993378    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/380c3494-b8be-4efd-b495-414a2819d883-istio-podinfo" pod "380c3494-b8be-4efd-b495-414a2819d883" (UID: "380c3494-b8be-4efd-b495-414a2819d883"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:07.994280 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.994243    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-kube-api-access-qf268" pod "67e4a918-26ba-4ebd-8822-a182e8c84418" (UID: "67e4a918-26ba-4ebd-8822-a182e8c84418"). InnerVolumeSpecName "kube-api-access-qf268". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:07.996797 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.996770    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-token" pod "67e4a918-26ba-4ebd-8822-a182e8c84418" (UID: "67e4a918-26ba-4ebd-8822-a182e8c84418"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:07.997255 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.997231    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-istio-token" pod "380c3494-b8be-4efd-b495-414a2819d883" (UID: "380c3494-b8be-4efd-b495-414a2819d883"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:07.997349 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:07.997263    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-kube-api-access-w4hr5" pod "380c3494-b8be-4efd-b495-414a2819d883" (UID: "380c3494-b8be-4efd-b495-414a2819d883"). InnerVolumeSpecName "kube-api-access-w4hr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:08.002210 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.002188    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-podinfo" pod "67e4a918-26ba-4ebd-8822-a182e8c84418" (UID: "67e4a918-26ba-4ebd-8822-a182e8c84418"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:08.088744 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088708    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.088744 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088745    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.088908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088761    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.088908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088777    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4hr5\" (UniqueName: \"kubernetes.io/projected/380c3494-b8be-4efd-b495-414a2819d883-kube-api-access-w4hr5\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.088908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088794    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.088908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088807    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/380c3494-b8be-4efd-b495-414a2819d883-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.088908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088823    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.088908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088837    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67e4a918-26ba-4ebd-8822-a182e8c84418-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.088908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088851    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.088908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088865    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.088908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088879    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.088908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088893    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.088908 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088905    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/380c3494-b8be-4efd-b495-414a2819d883-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.089264 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088919    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.089264 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088935    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qf268\" (UniqueName: \"kubernetes.io/projected/67e4a918-26ba-4ebd-8822-a182e8c84418-kube-api-access-qf268\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.089264 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088948    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/380c3494-b8be-4efd-b495-414a2819d883-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.089264 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.088962    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67e4a918-26ba-4ebd-8822-a182e8c84418-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.189259 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.189226    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"]
Oct 30 19:17:08.195678 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.195650    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-r6lh7"]
Oct 30 19:17:08.267351 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.267260    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:08.423147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.423083    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" path="/var/lib/kubelet/pods/227555e2-ac4e-4033-a23b-0c43c6eb37ac/volumes"
Oct 30 19:17:08.826027 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.825981    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws" podUID="84ce6a41-5d27-44aa-86bd-b589a00d0c2a" containerName="payload-pre-processing" containerID="cri-o://1b28838d2b76b0d9aa61d845716f30c37ca2f9e94c7dde9c35b56f4b4a767640" gracePeriod=25
Oct 30 19:17:08.892550 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.892508    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws" podUID="84ce6a41-5d27-44aa-86bd-b589a00d0c2a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:17:08.903744 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.903715    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"
Oct 30 19:17:08.903744 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.903734    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z" event={"ID":"67e4a918-26ba-4ebd-8822-a182e8c84418","Type":"ContainerDied","Data":"8d60d16ac7f9a09104b877eb0c4e0e55de639a4b3946e208a15404ebaae6be0e"}
Oct 30 19:17:08.903962 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.903795    4925 scope.go:122] "RemoveContainer" containerID="bcf6dff602e924504b405ee69603368a5cecb943ddb22666a0e762646d4a5090"
Oct 30 19:17:08.907566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.907542    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"
Oct 30 19:17:08.907566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.907553    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h" event={"ID":"380c3494-b8be-4efd-b495-414a2819d883","Type":"ContainerDied","Data":"cef4eb09042f6f01fcbfe25f8e1e252a8dbba7329d83e5be5afd89e09cad0631"}
Oct 30 19:17:08.914588 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.914563    4925 scope.go:122] "RemoveContainer" containerID="4740448450e61855e5c0676c85b49eb968b1c05cacfa6499201de2d7215a81ca"
Oct 30 19:17:08.947121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.944338    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"]
Oct 30 19:17:08.948647 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.948624    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-f486z"]
Oct 30 19:17:08.955489 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.955462    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"]
Oct 30 19:17:08.959050 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.959027    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-f3d6a5-openshift-default-79b6db5bf7-lzq4h"]
Oct 30 19:17:08.969120 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:08.969076    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:09.525450 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:09.525410    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:09.724429 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:09.724381    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:09.914938 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:09.914898    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:10.423511 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:10.423478    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="380c3494-b8be-4efd-b495-414a2819d883" path="/var/lib/kubelet/pods/380c3494-b8be-4efd-b495-414a2819d883/volumes"
Oct 30 19:17:10.423987 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:10.423970    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" path="/var/lib/kubelet/pods/67e4a918-26ba-4ebd-8822-a182e8c84418/volumes"
Oct 30 19:17:10.910929 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:10.910898    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:11.497955 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.497614    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:11.602589 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.602529    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 19:17:11.725686 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.725649    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:11.729763 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.729731    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-token\" (UniqueName: \"kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-token\") pod \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") "
Oct 30 19:17:11.729922 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.729802    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istiod-ca-cert\") pod \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") "
Oct 30 19:17:11.729922 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.729851    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-kube-api-access-mz4fp\" (UniqueName: \"kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-kube-api-access-mz4fp\") pod \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") "
Oct 30 19:17:11.730055 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.729917    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-socket\") pod \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") "
Oct 30 19:17:11.730055 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.729963    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-certs\") pod \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") "
Oct 30 19:17:11.730055 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730013    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-podinfo\") pod \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") "
Oct 30 19:17:11.730278 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730056    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-data\") pod \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") "
Oct 30 19:17:11.730278 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730166    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-envoy\") pod \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") "
Oct 30 19:17:11.730278 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730195    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-credential-socket\") pod \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\" (UID: \"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4\") "
Oct 30 19:17:11.730441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730294    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-socket" pod "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" (UID: "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:11.730441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730309    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istiod-ca-cert" pod "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" (UID: "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:11.730441 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730313    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-certs" pod "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" (UID: "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:11.730629 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730500    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-data" pod "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" (UID: "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:11.730691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730649    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:11.730737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730665    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:11.730737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730707    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:11.730737 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730716    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:11.730864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.730731    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-credential-socket" pod "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" (UID: "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:11.736152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.732971    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-token" pod "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" (UID: "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:11.736152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.733069    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-envoy" pod "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" (UID: "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:11.736494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.736471    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-kube-api-access-mz4fp" pod "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" (UID: "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4"). InnerVolumeSpecName "kube-api-access-mz4fp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:11.739217 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.739188    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-podinfo" pod "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" (UID: "bdb7e1fe-ff9f-456f-a913-8fef2ed627f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:11.831280 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.831257    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:11.831280 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.831279    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz4fp\" (UniqueName: \"kubernetes.io/projected/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-kube-api-access-mz4fp\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:11.831433 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.831290    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:11.831433 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.831299    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:11.831433 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.831308    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:11.928999 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.928965    4925 generic.go:356] "Generic (PLEG): container finished" podID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerID="8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84" exitCode=0
Oct 30 19:17:11.929411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.928998    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" event={"ID":"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4","Type":"ContainerDied","Data":"8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84"}
Oct 30 19:17:11.929411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.929022    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"
Oct 30 19:17:11.929411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.929056    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9" event={"ID":"bdb7e1fe-ff9f-456f-a913-8fef2ed627f4","Type":"ContainerDied","Data":"80515137f03a64db9ca354052ef809bc60793515d69101450d1c92c7bda4c298"}
Oct 30 19:17:11.929411 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.929081    4925 scope.go:122] "RemoveContainer" containerID="8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84"
Oct 30 19:17:11.939520 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.939504    4925 scope.go:122] "RemoveContainer" containerID="8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84"
Oct 30 19:17:11.939792 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:17:11.939774    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84\": container with ID starting with 8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84 not found: ID does not exist" containerID="8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84"
Oct 30 19:17:11.939864 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.939799    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84"} err="failed to get container status \"8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84\": rpc error: code = NotFound desc = could not find container \"8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84\": container with ID starting with 8402d44e751e0d016326cdbf297f8419094bdabca50a3287a2b64653c5aa2d84 not found: ID does not exist"
Oct 30 19:17:11.962892 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.962868    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"]
Oct 30 19:17:11.980789 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:11.980762    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-2889dd-openshift-default-84768d69f7-749x9"]
Oct 30 19:17:12.421474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:12.421444    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" path="/var/lib/kubelet/pods/bdb7e1fe-ff9f-456f-a913-8fef2ed627f4/volumes"
Oct 30 19:17:12.744174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:12.744084    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:13.029464 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:13.029381    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:13.305785 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:17:13.305697    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:17:13.533437 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:13.533407    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:13.876124 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:13.876062    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:14.122905 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:14.122866    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:14.318552 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:14.318515    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:14.489615 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:14.489571    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw" podUID="f5fb074d-ef9f-44e0-92bd-5e4ab059b78e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:17:15.406663 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.406633    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:15.501493 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:17:15.501471    4925 logging.go:55] [core] [Channel #9345 SubChannel #9346]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.207:50052", ServerName: "10.132.0.207:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.207:50052: connect: connection refused"
Oct 30 19:17:15.674089 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.674062    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"
Oct 30 19:17:15.703398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.703371    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"
Oct 30 19:17:15.775810 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.775777    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57195712-20ec-4a9b-97ce-bfc531287e3a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57195712-20ec-4a9b-97ce-bfc531287e3a-plugins-config-volume\") pod \"57195712-20ec-4a9b-97ce-bfc531287e3a\" (UID: \"57195712-20ec-4a9b-97ce-bfc531287e3a\") "
Oct 30 19:17:15.776062 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.775921    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57195712-20ec-4a9b-97ce-bfc531287e3a-kube-api-access-d227w\" (UniqueName: \"kubernetes.io/projected/57195712-20ec-4a9b-97ce-bfc531287e3a-kube-api-access-d227w\") pod \"57195712-20ec-4a9b-97ce-bfc531287e3a\" (UID: \"57195712-20ec-4a9b-97ce-bfc531287e3a\") "
Oct 30 19:17:15.776572 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.776531    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57195712-20ec-4a9b-97ce-bfc531287e3a-plugins-config-volume" pod "57195712-20ec-4a9b-97ce-bfc531287e3a" (UID: "57195712-20ec-4a9b-97ce-bfc531287e3a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:15.778501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.778476    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57195712-20ec-4a9b-97ce-bfc531287e3a-kube-api-access-d227w" pod "57195712-20ec-4a9b-97ce-bfc531287e3a" (UID: "57195712-20ec-4a9b-97ce-bfc531287e3a"). InnerVolumeSpecName "kube-api-access-d227w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:15.877281 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.877213    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-plugins-config-volume\") pod \"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e\" (UID: \"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e\") "
Oct 30 19:17:15.877464 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.877327    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-kube-api-access-4c8bj\" (UniqueName: \"kubernetes.io/projected/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-kube-api-access-4c8bj\") pod \"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e\" (UID: \"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e\") "
Oct 30 19:17:15.877698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.877673    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d227w\" (UniqueName: \"kubernetes.io/projected/57195712-20ec-4a9b-97ce-bfc531287e3a-kube-api-access-d227w\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:15.877698 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.877698    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57195712-20ec-4a9b-97ce-bfc531287e3a-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:15.877867 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.877813    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-plugins-config-volume" pod "f5fb074d-ef9f-44e0-92bd-5e4ab059b78e" (UID: "f5fb074d-ef9f-44e0-92bd-5e4ab059b78e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:15.880008 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.879979    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-kube-api-access-4c8bj" pod "f5fb074d-ef9f-44e0-92bd-5e4ab059b78e" (UID: "f5fb074d-ef9f-44e0-92bd-5e4ab059b78e"). InnerVolumeSpecName "kube-api-access-4c8bj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:15.957562 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.957532    4925 generic.go:356] "Generic (PLEG): container finished" podID="f5fb074d-ef9f-44e0-92bd-5e4ab059b78e" containerID="3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63" exitCode=0
Oct 30 19:17:15.957711 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.957592    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"
Oct 30 19:17:15.957711 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.957622    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw" event={"ID":"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e","Type":"ContainerDied","Data":"3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63"}
Oct 30 19:17:15.957846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.957713    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw" event={"ID":"f5fb074d-ef9f-44e0-92bd-5e4ab059b78e","Type":"ContainerDied","Data":"290bb0a56a753958265b2275cc2e74475a6dfe2941141fbc5cfab5147730bde1"}
Oct 30 19:17:15.957846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.957744    4925 scope.go:122] "RemoveContainer" containerID="3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63"
Oct 30 19:17:15.959447 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.959401    4925 generic.go:356] "Generic (PLEG): container finished" podID="57195712-20ec-4a9b-97ce-bfc531287e3a" containerID="07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4" exitCode=0
Oct 30 19:17:15.959557 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.959485    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv" event={"ID":"57195712-20ec-4a9b-97ce-bfc531287e3a","Type":"ContainerDied","Data":"07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4"}
Oct 30 19:17:15.959557 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.959531    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv" event={"ID":"57195712-20ec-4a9b-97ce-bfc531287e3a","Type":"ContainerDied","Data":"540395b15a79552cbe510f194f7f4b774259d5fdd671179702aca0f5ef9ef646"}
Oct 30 19:17:15.959676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.959661    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"
Oct 30 19:17:15.969246 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.969166    4925 scope.go:122] "RemoveContainer" containerID="3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63"
Oct 30 19:17:15.969555 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:17:15.969528    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63\": container with ID starting with 3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63 not found: ID does not exist" containerID="3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63"
Oct 30 19:17:15.969625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.969563    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63"} err="failed to get container status \"3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63\": rpc error: code = NotFound desc = could not find container \"3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63\": container with ID starting with 3180e8dbd4d33ae0491da2c915fbb61870835989d4b955f3dec6eecb7d035e63 not found: ID does not exist"
Oct 30 19:17:15.969625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.969584    4925 scope.go:122] "RemoveContainer" containerID="07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4"
Oct 30 19:17:15.978764 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.978741    4925 scope.go:122] "RemoveContainer" containerID="07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4"
Oct 30 19:17:15.978873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.978803    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:15.978873 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.978827    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c8bj\" (UniqueName: \"kubernetes.io/projected/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e-kube-api-access-4c8bj\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:15.979057 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:17:15.979041    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4\": container with ID starting with 07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4 not found: ID does not exist" containerID="07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4"
Oct 30 19:17:15.979164 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.979062    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4"} err="failed to get container status \"07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4\": rpc error: code = NotFound desc = could not find container \"07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4\": container with ID starting with 07be15793954fea54aaaf71b9dca7bfa7204b128bdd2447729a1a83911bf97f4 not found: ID does not exist"
Oct 30 19:17:15.985847 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.985827    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"]
Oct 30 19:17:15.989802 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.989781    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-f3d6a5-5c85fcfbff-nxprw"]
Oct 30 19:17:15.999919 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:15.999899    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"]
Oct 30 19:17:16.003323 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:16.003301    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv"]
Oct 30 19:17:16.171990 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:16.171949    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:16.421317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:16.421288    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57195712-20ec-4a9b-97ce-bfc531287e3a" path="/var/lib/kubelet/pods/57195712-20ec-4a9b-97ce-bfc531287e3a/volumes"
Oct 30 19:17:16.421692 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:16.421647    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5fb074d-ef9f-44e0-92bd-5e4ab059b78e" path="/var/lib/kubelet/pods/f5fb074d-ef9f-44e0-92bd-5e4ab059b78e/volumes"
Oct 30 19:17:16.501107 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:16.501060    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-f3d6a5-75b5cb7b4d-grxqv" podUID="57195712-20ec-4a9b-97ce-bfc531287e3a" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.207:50052\" within 1s: context deadline exceeded"
Oct 30 19:17:16.978760 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:16.978726    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:17.191188 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:17.190936    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:17.442121 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:17.437253    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:18.031221 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:18.031187    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:18.220940 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:18.220907    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:18.434533 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:18.434502    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:18.892701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:18.892608    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws" podUID="84ce6a41-5d27-44aa-86bd-b589a00d0c2a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:17:18.995484 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:18.995443    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:19.427139 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:19.427089    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:19.687815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:19.687719    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:20.395625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:20.395588    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:21.212676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:21.212640    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:21.932567 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:21.932532    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:23.102774 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:23.102728    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:24.006009 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:24.005972    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:24.781915 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:24.781878    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:25.479953 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:25.479914    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:25.801218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:25.801088    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:26.033031 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:26.032998    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"]
Oct 30 19:17:26.033658 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:26.033621    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" podUID="60687fb8-b28d-474d-9e88-7d919d962b79" containerName="istio-proxy" containerID="cri-o://a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2" gracePeriod=30
Oct 30 19:17:26.052367 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:26.052286    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:26.351272 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:26.350585    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:26.518893 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:26.518859    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"]
Oct 30 19:17:26.519139 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:26.519111    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp" podUID="e449af3e-0d2b-4084-a82e-34752431dae9" containerName="istio-proxy" containerID="cri-o://4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a" gracePeriod=30
Oct 30 19:17:26.683165 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:26.683131    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:26.877711 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:26.877673    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:27.104451 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:27.104356    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:28.006914 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:28.006873    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:28.887039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:28.887002    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:28.891175 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:17:28.891154    4925 logging.go:55] [core] [Channel #9359 SubChannel #9360]grpc: addrConn.createTransport failed to connect to {Addr: "10.132.0.54:50052", ServerName: "10.132.0.54:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.132.0.54:50052: connect: connection refused"
Oct 30 19:17:29.076714 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:29.076603    4925 generic.go:356] "Generic (PLEG): container finished" podID="84ce6a41-5d27-44aa-86bd-b589a00d0c2a" containerID="1b28838d2b76b0d9aa61d845716f30c37ca2f9e94c7dde9c35b56f4b4a767640" exitCode=0
Oct 30 19:17:29.076714 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:29.076678    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws" event={"ID":"84ce6a41-5d27-44aa-86bd-b589a00d0c2a","Type":"ContainerDied","Data":"1b28838d2b76b0d9aa61d845716f30c37ca2f9e94c7dde9c35b56f4b4a767640"}
Oct 30 19:17:29.096928 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:29.096865    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 19:17:29.133713 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:29.133677    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-kube-api-access-7fkls\" (UniqueName: \"kubernetes.io/projected/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-kube-api-access-7fkls\") pod \"84ce6a41-5d27-44aa-86bd-b589a00d0c2a\" (UID: \"84ce6a41-5d27-44aa-86bd-b589a00d0c2a\") "
Oct 30 19:17:29.133874 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:29.133856    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-plugins-config-volume\") pod \"84ce6a41-5d27-44aa-86bd-b589a00d0c2a\" (UID: \"84ce6a41-5d27-44aa-86bd-b589a00d0c2a\") "
Oct 30 19:17:29.134239 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:29.134216    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-plugins-config-volume" pod "84ce6a41-5d27-44aa-86bd-b589a00d0c2a" (UID: "84ce6a41-5d27-44aa-86bd-b589a00d0c2a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:29.136162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:29.136125    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-kube-api-access-7fkls" pod "84ce6a41-5d27-44aa-86bd-b589a00d0c2a" (UID: "84ce6a41-5d27-44aa-86bd-b589a00d0c2a"). InnerVolumeSpecName "kube-api-access-7fkls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:29.234833 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:29.234801    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fkls\" (UniqueName: \"kubernetes.io/projected/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-kube-api-access-7fkls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:29.234833 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:29.234828    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84ce6a41-5d27-44aa-86bd-b589a00d0c2a-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:29.580396 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:29.580362    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:29.891343 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:29.891244    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws" podUID="84ce6a41-5d27-44aa-86bd-b589a00d0c2a" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.132.0.54:50052\" within 1s: context deadline exceeded"
Oct 30 19:17:29.891501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:29.891353    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 19:17:30.083701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:30.083667    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws" event={"ID":"84ce6a41-5d27-44aa-86bd-b589a00d0c2a","Type":"ContainerDied","Data":"955b99816732d99434447e17cd336a9e768020f939d9528762aeb78a95f6cc54"}
Oct 30 19:17:30.083701 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:30.083702    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"
Oct 30 19:17:30.084260 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:30.083736    4925 scope.go:122] "RemoveContainer" containerID="1b28838d2b76b0d9aa61d845716f30c37ca2f9e94c7dde9c35b56f4b4a767640"
Oct 30 19:17:30.107770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:30.107744    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"]
Oct 30 19:17:30.110904 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:30.110879    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-39bde3-754c6b4995-v6vws"]
Oct 30 19:17:30.282979 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:30.282945    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:30.421336 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:30.421305    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="84ce6a41-5d27-44aa-86bd-b589a00d0c2a" path="/var/lib/kubelet/pods/84ce6a41-5d27-44aa-86bd-b589a00d0c2a/volumes"
Oct 30 19:17:30.994599 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:30.994561    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:31.323513 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.323488    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 19:17:31.363524 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.363492    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-envoy\") pod \"60687fb8-b28d-474d-9e88-7d919d962b79\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") "
Oct 30 19:17:31.363694 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.363535    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-socket\") pod \"60687fb8-b28d-474d-9e88-7d919d962b79\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") "
Oct 30 19:17:31.363694 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.363597    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-credential-socket\") pod \"60687fb8-b28d-474d-9e88-7d919d962b79\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") "
Oct 30 19:17:31.363694 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.363619    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-kube-api-access-p9xkx\" (UniqueName: \"kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-kube-api-access-p9xkx\") pod \"60687fb8-b28d-474d-9e88-7d919d962b79\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") "
Oct 30 19:17:31.363694 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.363637    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60687fb8-b28d-474d-9e88-7d919d962b79-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60687fb8-b28d-474d-9e88-7d919d962b79-istiod-ca-cert\") pod \"60687fb8-b28d-474d-9e88-7d919d962b79\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") "
Oct 30 19:17:31.363694 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.363687    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-certs\") pod \"60687fb8-b28d-474d-9e88-7d919d962b79\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") "
Oct 30 19:17:31.363978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.363911    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-socket" pod "60687fb8-b28d-474d-9e88-7d919d962b79" (UID: "60687fb8-b28d-474d-9e88-7d919d962b79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:31.363978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.363946    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-credential-socket" pod "60687fb8-b28d-474d-9e88-7d919d962b79" (UID: "60687fb8-b28d-474d-9e88-7d919d962b79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:31.364114 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.363982    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-certs" pod "60687fb8-b28d-474d-9e88-7d919d962b79" (UID: "60687fb8-b28d-474d-9e88-7d919d962b79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:31.364114 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.364007    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/60687fb8-b28d-474d-9e88-7d919d962b79-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60687fb8-b28d-474d-9e88-7d919d962b79-istio-podinfo\") pod \"60687fb8-b28d-474d-9e88-7d919d962b79\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") "
Oct 30 19:17:31.364238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.364129    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-data\") pod \"60687fb8-b28d-474d-9e88-7d919d962b79\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") "
Oct 30 19:17:31.364238 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.364166    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-istio-token\" (UniqueName: \"kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-istio-token\") pod \"60687fb8-b28d-474d-9e88-7d919d962b79\" (UID: \"60687fb8-b28d-474d-9e88-7d919d962b79\") "
Oct 30 19:17:31.364359 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.364246    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60687fb8-b28d-474d-9e88-7d919d962b79-istiod-ca-cert" pod "60687fb8-b28d-474d-9e88-7d919d962b79" (UID: "60687fb8-b28d-474d-9e88-7d919d962b79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:31.364529 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.364509    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-data" pod "60687fb8-b28d-474d-9e88-7d919d962b79" (UID: "60687fb8-b28d-474d-9e88-7d919d962b79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:31.364643 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.364625    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.364706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.364650    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60687fb8-b28d-474d-9e88-7d919d962b79-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.364706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.364664    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.364706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.364678    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.364706 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.364691    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.366398 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.366371    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-kube-api-access-p9xkx" pod "60687fb8-b28d-474d-9e88-7d919d962b79" (UID: "60687fb8-b28d-474d-9e88-7d919d962b79"). InnerVolumeSpecName "kube-api-access-p9xkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:31.367372 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.367342    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60687fb8-b28d-474d-9e88-7d919d962b79-istio-podinfo" pod "60687fb8-b28d-474d-9e88-7d919d962b79" (UID: "60687fb8-b28d-474d-9e88-7d919d962b79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:31.367486 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.367446    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-envoy" pod "60687fb8-b28d-474d-9e88-7d919d962b79" (UID: "60687fb8-b28d-474d-9e88-7d919d962b79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:31.368819 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.368798    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-istio-token" pod "60687fb8-b28d-474d-9e88-7d919d962b79" (UID: "60687fb8-b28d-474d-9e88-7d919d962b79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:31.465848 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.465814    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60687fb8-b28d-474d-9e88-7d919d962b79-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.465848 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.465848    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.466111 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.465861    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60687fb8-b28d-474d-9e88-7d919d962b79-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.466111 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.465875    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9xkx\" (UniqueName: \"kubernetes.io/projected/60687fb8-b28d-474d-9e88-7d919d962b79-kube-api-access-p9xkx\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.781633 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.781599    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:17:31.792059 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.792030    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:31.869876 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.869785    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-credential-socket\") pod \"e449af3e-0d2b-4084-a82e-34752431dae9\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") "
Oct 30 19:17:31.869876 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.869859    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-data\") pod \"e449af3e-0d2b-4084-a82e-34752431dae9\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") "
Oct 30 19:17:31.870117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.869927    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-socket\") pod \"e449af3e-0d2b-4084-a82e-34752431dae9\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") "
Oct 30 19:17:31.870117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.869975    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e449af3e-0d2b-4084-a82e-34752431dae9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e449af3e-0d2b-4084-a82e-34752431dae9-istio-podinfo\") pod \"e449af3e-0d2b-4084-a82e-34752431dae9\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") "
Oct 30 19:17:31.870117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.870005    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-envoy\") pod \"e449af3e-0d2b-4084-a82e-34752431dae9\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") "
Oct 30 19:17:31.870117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.870056    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-istio-token\" (UniqueName: \"kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-istio-token\") pod \"e449af3e-0d2b-4084-a82e-34752431dae9\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") "
Oct 30 19:17:31.870117 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.870084    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e449af3e-0d2b-4084-a82e-34752431dae9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e449af3e-0d2b-4084-a82e-34752431dae9-istiod-ca-cert\") pod \"e449af3e-0d2b-4084-a82e-34752431dae9\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") "
Oct 30 19:17:31.870407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.870158    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-kube-api-access-mthdq\" (UniqueName: \"kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-kube-api-access-mthdq\") pod \"e449af3e-0d2b-4084-a82e-34752431dae9\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") "
Oct 30 19:17:31.870407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.870177    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-credential-socket" pod "e449af3e-0d2b-4084-a82e-34752431dae9" (UID: "e449af3e-0d2b-4084-a82e-34752431dae9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:31.870407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.870193    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-certs\") pod \"e449af3e-0d2b-4084-a82e-34752431dae9\" (UID: \"e449af3e-0d2b-4084-a82e-34752431dae9\") "
Oct 30 19:17:31.870407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.870208    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-data" pod "e449af3e-0d2b-4084-a82e-34752431dae9" (UID: "e449af3e-0d2b-4084-a82e-34752431dae9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:31.870625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.870550    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e449af3e-0d2b-4084-a82e-34752431dae9-istiod-ca-cert" pod "e449af3e-0d2b-4084-a82e-34752431dae9" (UID: "e449af3e-0d2b-4084-a82e-34752431dae9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:31.870625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.870565    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.870625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.870584    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.870789 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.870631    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-socket" pod "e449af3e-0d2b-4084-a82e-34752431dae9" (UID: "e449af3e-0d2b-4084-a82e-34752431dae9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:31.871162 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.871088    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-certs" pod "e449af3e-0d2b-4084-a82e-34752431dae9" (UID: "e449af3e-0d2b-4084-a82e-34752431dae9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:31.873060 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.873032    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-kube-api-access-mthdq" pod "e449af3e-0d2b-4084-a82e-34752431dae9" (UID: "e449af3e-0d2b-4084-a82e-34752431dae9"). InnerVolumeSpecName "kube-api-access-mthdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:31.873212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.873179    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-istio-token" pod "e449af3e-0d2b-4084-a82e-34752431dae9" (UID: "e449af3e-0d2b-4084-a82e-34752431dae9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:31.873293 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.873274    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e449af3e-0d2b-4084-a82e-34752431dae9-istio-podinfo" pod "e449af3e-0d2b-4084-a82e-34752431dae9" (UID: "e449af3e-0d2b-4084-a82e-34752431dae9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:31.873293 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.873281    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-envoy" pod "e449af3e-0d2b-4084-a82e-34752431dae9" (UID: "e449af3e-0d2b-4084-a82e-34752431dae9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:31.971810 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.971774    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mthdq\" (UniqueName: \"kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-kube-api-access-mthdq\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.971810 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.971800    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.971810 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.971809    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.972057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.971819    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e449af3e-0d2b-4084-a82e-34752431dae9-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.972057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.971828    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e449af3e-0d2b-4084-a82e-34752431dae9-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.972057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.971836    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e449af3e-0d2b-4084-a82e-34752431dae9-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:31.972057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:31.971844    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e449af3e-0d2b-4084-a82e-34752431dae9-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:32.098439 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.098407    4925 generic.go:356] "Generic (PLEG): container finished" podID="e449af3e-0d2b-4084-a82e-34752431dae9" containerID="4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a" exitCode=0
Oct 30 19:17:32.098584 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.098446    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp" event={"ID":"e449af3e-0d2b-4084-a82e-34752431dae9","Type":"ContainerDied","Data":"4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a"}
Oct 30 19:17:32.098584 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.098487    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"
Oct 30 19:17:32.098584 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.098504    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp" event={"ID":"e449af3e-0d2b-4084-a82e-34752431dae9","Type":"ContainerDied","Data":"78f80493e8096d872dc635d1352cc9f48d80b7594f675355f0e16ede4267e88b"}
Oct 30 19:17:32.098584 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.098532    4925 scope.go:122] "RemoveContainer" containerID="4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a"
Oct 30 19:17:32.103629 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.100087    4925 generic.go:356] "Generic (PLEG): container finished" podID="60687fb8-b28d-474d-9e88-7d919d962b79" containerID="a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2" exitCode=0
Oct 30 19:17:32.103629 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.100145    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" event={"ID":"60687fb8-b28d-474d-9e88-7d919d962b79","Type":"ContainerDied","Data":"a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2"}
Oct 30 19:17:32.103629 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.100183    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9" event={"ID":"60687fb8-b28d-474d-9e88-7d919d962b79","Type":"ContainerDied","Data":"af127686693997eae30b7a2c39d73daf33d15659394b981782314e290ccfe78f"}
Oct 30 19:17:32.103629 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.100249    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"
Oct 30 19:17:32.112278 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.112262    4925 scope.go:122] "RemoveContainer" containerID="4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a"
Oct 30 19:17:32.112521 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:17:32.112503    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a\": container with ID starting with 4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a not found: ID does not exist" containerID="4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a"
Oct 30 19:17:32.112597 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.112534    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a"} err="failed to get container status \"4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a\": rpc error: code = NotFound desc = could not find container \"4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a\": container with ID starting with 4b03c82d1d8e3b39ee407aab9d54911709638de6441729378ca71df38254169a not found: ID does not exist"
Oct 30 19:17:32.112597 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.112556    4925 scope.go:122] "RemoveContainer" containerID="a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2"
Oct 30 19:17:32.121928 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.121903    4925 scope.go:122] "RemoveContainer" containerID="a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2"
Oct 30 19:17:32.122249 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:17:32.122230    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2\": container with ID starting with a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2 not found: ID does not exist" containerID="a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2"
Oct 30 19:17:32.122334 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.122256    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2"} err="failed to get container status \"a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2\": rpc error: code = NotFound desc = could not find container \"a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2\": container with ID starting with a609373ee609bedaf1e330b6ce0715534f87ddc7745180f92dbaf5de9ca137f2 not found: ID does not exist"
Oct 30 19:17:32.132651 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.132630    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"]
Oct 30 19:17:32.134414 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.134394    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-dcd326-openshift-default-d787c4f6f-86bpp"]
Oct 30 19:17:32.150218 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.150201    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"]
Oct 30 19:17:32.161290 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.161270    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-39bde3-openshift-default-5d45cdbd49-vjhf9"]
Oct 30 19:17:32.421895 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.421865    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60687fb8-b28d-474d-9e88-7d919d962b79" path="/var/lib/kubelet/pods/60687fb8-b28d-474d-9e88-7d919d962b79/volumes"
Oct 30 19:17:32.422490 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.422474    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e449af3e-0d2b-4084-a82e-34752431dae9" path="/var/lib/kubelet/pods/e449af3e-0d2b-4084-a82e-34752431dae9/volumes"
Oct 30 19:17:32.642021 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:32.641985    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:33.545577 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:33.545532    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:34.294914 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:34.294877    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:35.025992 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:35.025955    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:35.697582 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:35.697548    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:36.239452 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:36.239413    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:36.479676 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:36.479618    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:36.680123 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:36.680074    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:37.359797 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:37.359763    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:37.564561 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:37.564526    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:38.096978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:38.096943    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:38.283947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:38.283903    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:38.932212 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:38.932170    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:39.375685 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.375602    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"]
Oct 30 19:17:39.376168 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.376117    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c" podUID="5906e228-28c9-4e6d-9a10-c4336f9d133d" containerName="maas-api" containerID="cri-o://2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5" gracePeriod=30
Oct 30 19:17:39.510584 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.510543    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:39.677599 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.677572    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:17:39.711341 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.711268    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:39.752854 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.752826    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-metrics-certs\") pod \"5906e228-28c9-4e6d-9a10-c4336f9d133d\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") "
Oct 30 19:17:39.753010 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.752873    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5906e228-28c9-4e6d-9a10-c4336f9d133d-kube-api-access-lvd7h\" (UniqueName: \"kubernetes.io/projected/5906e228-28c9-4e6d-9a10-c4336f9d133d-kube-api-access-lvd7h\") pod \"5906e228-28c9-4e6d-9a10-c4336f9d133d\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") "
Oct 30 19:17:39.753010 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.752991    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-maas-api-tls\") pod \"5906e228-28c9-4e6d-9a10-c4336f9d133d\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") "
Oct 30 19:17:39.753136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.753072    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5906e228-28c9-4e6d-9a10-c4336f9d133d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5906e228-28c9-4e6d-9a10-c4336f9d133d-openshift-service-ca\") pod \"5906e228-28c9-4e6d-9a10-c4336f9d133d\" (UID: \"5906e228-28c9-4e6d-9a10-c4336f9d133d\") "
Oct 30 19:17:39.753739 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.753538    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5906e228-28c9-4e6d-9a10-c4336f9d133d-openshift-service-ca" pod "5906e228-28c9-4e6d-9a10-c4336f9d133d" (UID: "5906e228-28c9-4e6d-9a10-c4336f9d133d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:39.755165 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.755140    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5906e228-28c9-4e6d-9a10-c4336f9d133d-kube-api-access-lvd7h" pod "5906e228-28c9-4e6d-9a10-c4336f9d133d" (UID: "5906e228-28c9-4e6d-9a10-c4336f9d133d"). InnerVolumeSpecName "kube-api-access-lvd7h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.755285 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.755184    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-maas-api-tls" pod "5906e228-28c9-4e6d-9a10-c4336f9d133d" (UID: "5906e228-28c9-4e6d-9a10-c4336f9d133d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:39.755285 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.755195    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-metrics-certs" pod "5906e228-28c9-4e6d-9a10-c4336f9d133d" (UID: "5906e228-28c9-4e6d-9a10-c4336f9d133d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:39.854028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.853994    4925 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-metrics-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.854028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.854026    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvd7h\" (UniqueName: \"kubernetes.io/projected/5906e228-28c9-4e6d-9a10-c4336f9d133d-kube-api-access-lvd7h\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.854028 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.854037    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5906e228-28c9-4e6d-9a10-c4336f9d133d-maas-api-tls\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.854299 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.854046    4925 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5906e228-28c9-4e6d-9a10-c4336f9d133d-openshift-service-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.891634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:39.891590    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:40.147820 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:40.147784    4925 generic.go:356] "Generic (PLEG): container finished" podID="5906e228-28c9-4e6d-9a10-c4336f9d133d" containerID="2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5" exitCode=0
Oct 30 19:17:40.148281 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:40.147967    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c" event={"ID":"5906e228-28c9-4e6d-9a10-c4336f9d133d","Type":"ContainerDied","Data":"2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5"}
Oct 30 19:17:40.148281 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:40.148017    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c" event={"ID":"5906e228-28c9-4e6d-9a10-c4336f9d133d","Type":"ContainerDied","Data":"984294bd9f1d95937d10edadd7aaf78a059ab09f64fbfca82b131e03b46a64bc"}
Oct 30 19:17:40.148281 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:40.148045    4925 scope.go:122] "RemoveContainer" containerID="2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5"
Oct 30 19:17:40.148281 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:40.148237    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"
Oct 30 19:17:40.164322 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:40.164215    4925 scope.go:122] "RemoveContainer" containerID="2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5"
Oct 30 19:17:40.164554 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:17:40.164529    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5\": container with ID starting with 2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5 not found: ID does not exist" containerID="2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5"
Oct 30 19:17:40.164639 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:40.164567    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5"} err="failed to get container status \"2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5\": rpc error: code = NotFound desc = could not find container \"2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5\": container with ID starting with 2d8effeba0243791e25ba72a6d21f1f342b9d8ce5f7c31438f3a85d53502b8b5 not found: ID does not exist"
Oct 30 19:17:40.176136 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:40.176087    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"]
Oct 30 19:17:40.182664 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:40.182638    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-46385c-55b796ccbd-87n7c"]
Oct 30 19:17:40.420729 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:40.420702    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5906e228-28c9-4e6d-9a10-c4336f9d133d" path="/var/lib/kubelet/pods/5906e228-28c9-4e6d-9a10-c4336f9d133d/volumes"
Oct 30 19:17:40.544847 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:40.544788    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:40.738786 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:40.738694    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:41.378672 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:41.378634    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:42.173362 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:42.173317    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:42.883543 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:42.883504    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:43.302798 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:17:43.302761    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:17:43.804468 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:43.804425    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:44.627604 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:44.627563    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:44.851912 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:44.851873    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:45.303813 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:45.303770    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:45.999199 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:45.999155    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:46.773313 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:46.773277    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:46.954371 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:46.954331    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:47.291569 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:47.291515    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:47.499426 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:47.499394    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:47.814133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:47.814079    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:48.436571 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:48.436538    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:49.122012 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:49.121976    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:49.946049 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:49.946012    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:50.590524 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:50.590492    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:51.303626 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:51.303594    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:52.005039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:52.004998    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:52.674409 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:52.674374    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"]
Oct 30 19:17:52.703691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:52.703655    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:52.889273 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:52.889240    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:53.468753 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:53.468721    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:54.214874 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:54.214837    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:54.870163 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:54.870125    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:55.564567 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:55.564534    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:56.175756 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:56.175717    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:56.863236 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:56.863205    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:57.464458 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:57.464427    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:57.988223 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:57.988188    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"]
Oct 30 19:17:58.158894 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.158862    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:58.336307 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.336218    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:58.647052 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.646965    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"]
Oct 30 19:17:58.647474 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647453    4925 cpu_manager.go:414] "containerMap: removing container" podUID="d2eed241-cb88-48e4-9cba-38e63d153cc3" containerName="revoke-keys"
Oct 30 19:17:58.647586 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647475    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2eed241-cb88-48e4-9cba-38e63d153cc3" containerName="revoke-keys"
Oct 30 19:17:58.647586 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647495    4925 cpu_manager.go:414] "containerMap: removing container" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerName="istio-proxy"
Oct 30 19:17:58.647586 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647502    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerName="istio-proxy"
Oct 30 19:17:58.647586 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647523    4925 cpu_manager.go:414] "containerMap: removing container" podUID="380c3494-b8be-4efd-b495-414a2819d883" containerName="istio-proxy"
Oct 30 19:17:58.647586 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647530    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="380c3494-b8be-4efd-b495-414a2819d883" containerName="istio-proxy"
Oct 30 19:17:58.647586 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647550    4925 cpu_manager.go:414] "containerMap: removing container" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy"
Oct 30 19:17:58.647586 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647557    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy"
Oct 30 19:17:58.647586 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647566    4925 cpu_manager.go:414] "containerMap: removing container" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerName="istio-proxy"
Oct 30 19:17:58.647586 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647573    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerName="istio-proxy"
Oct 30 19:17:58.647586 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647584    4925 cpu_manager.go:414] "containerMap: removing container" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647591    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647603    4925 cpu_manager.go:414] "containerMap: removing container" podUID="e449af3e-0d2b-4084-a82e-34752431dae9" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647610    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e449af3e-0d2b-4084-a82e-34752431dae9" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647624    4925 cpu_manager.go:414] "containerMap: removing container" podUID="5906e228-28c9-4e6d-9a10-c4336f9d133d" containerName="maas-api"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647631    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5906e228-28c9-4e6d-9a10-c4336f9d133d" containerName="maas-api"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647645    4925 cpu_manager.go:414] "containerMap: removing container" podUID="50839431-baf0-4a66-a775-54be5ea96723" containerName="cleanup"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647652    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50839431-baf0-4a66-a775-54be5ea96723" containerName="cleanup"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647669    4925 cpu_manager.go:414] "containerMap: removing container" podUID="57195712-20ec-4a9b-97ce-bfc531287e3a" containerName="payload-processing"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647676    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57195712-20ec-4a9b-97ce-bfc531287e3a" containerName="payload-processing"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647689    4925 cpu_manager.go:414] "containerMap: removing container" podUID="60687fb8-b28d-474d-9e88-7d919d962b79" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647698    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60687fb8-b28d-474d-9e88-7d919d962b79" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647711    4925 cpu_manager.go:414] "containerMap: removing container" podUID="84ce6a41-5d27-44aa-86bd-b589a00d0c2a" containerName="payload-pre-processing"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647718    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="84ce6a41-5d27-44aa-86bd-b589a00d0c2a" containerName="payload-pre-processing"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647732    4925 cpu_manager.go:414] "containerMap: removing container" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647740    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647749    4925 cpu_manager.go:414] "containerMap: removing container" podUID="f5fb074d-ef9f-44e0-92bd-5e4ab059b78e" containerName="payload-pre-processing"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647756    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5fb074d-ef9f-44e0-92bd-5e4ab059b78e" containerName="payload-pre-processing"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647862    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="380c3494-b8be-4efd-b495-414a2819d883" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647879    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="67e4a918-26ba-4ebd-8822-a182e8c84418" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647887    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="227555e2-ac4e-4033-a23b-0c43c6eb37ac" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647897    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="e449af3e-0d2b-4084-a82e-34752431dae9" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647907    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5760e51-b421-4e2b-8896-e61d63ef0e18" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647915    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="5906e228-28c9-4e6d-9a10-c4336f9d133d" containerName="maas-api"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647925    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="71f8aae7-be84-42e4-b3f3-66d95a9e5579" containerName="cleanup"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647951    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="57195712-20ec-4a9b-97ce-bfc531287e3a" containerName="payload-processing"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647961    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="60687fb8-b28d-474d-9e88-7d919d962b79" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647970    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="84ce6a41-5d27-44aa-86bd-b589a00d0c2a" containerName="payload-pre-processing"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647979    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdb7e1fe-ff9f-456f-a913-8fef2ed627f4" containerName="istio-proxy"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647987    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5fb074d-ef9f-44e0-92bd-5e4ab059b78e" containerName="payload-pre-processing"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.647997    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2eed241-cb88-48e4-9cba-38e63d153cc3" containerName="revoke-keys"
Oct 30 19:17:58.648158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.648007    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f91ff3f-7f4b-413b-a657-df0a3e19d1d5" containerName="istio-proxy"
Oct 30 19:17:58.662625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.662594    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"]
Oct 30 19:17:58.662803 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.662687    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:17:58.686593 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.686567    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:17:58.686759 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.686612    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-94hcn\""
Oct 30 19:17:58.853277 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.853238    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghkkl\" (UniqueName: \"kubernetes.io/projected/972c42f1-2638-4521-9122-aa6ebad53891-kube-api-access-ghkkl\") pod \"maas-api-revoke-sub-3190c194-w59ls\" (UID: \"972c42f1-2638-4521-9122-aa6ebad53891\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:17:58.853464 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.853281    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3190c194-w59ls\" (UID: \"972c42f1-2638-4521-9122-aa6ebad53891\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:17:58.853464 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.853435    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3190c194-w59ls\" (UID: \"972c42f1-2638-4521-9122-aa6ebad53891\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:17:58.954358 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.954328    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3190c194-w59ls\" (UID: \"972c42f1-2638-4521-9122-aa6ebad53891\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:17:58.954500 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.954367    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghkkl\" (UniqueName: \"kubernetes.io/projected/972c42f1-2638-4521-9122-aa6ebad53891-kube-api-access-ghkkl\") pod \"maas-api-revoke-sub-3190c194-w59ls\" (UID: \"972c42f1-2638-4521-9122-aa6ebad53891\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:17:58.954500 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.954393    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3190c194-w59ls\" (UID: \"972c42f1-2638-4521-9122-aa6ebad53891\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:17:58.954964 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.954945    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3190c194-w59ls\" (UID: \"972c42f1-2638-4521-9122-aa6ebad53891\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:17:58.956814 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.956796    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3190c194-w59ls\" (UID: \"972c42f1-2638-4521-9122-aa6ebad53891\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:17:58.962519 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.962494    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghkkl\" (UniqueName: \"kubernetes.io/projected/972c42f1-2638-4521-9122-aa6ebad53891-kube-api-access-ghkkl\") pod \"maas-api-revoke-sub-3190c194-w59ls\" (UID: \"972c42f1-2638-4521-9122-aa6ebad53891\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:17:58.979359 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:58.979328    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:17:59.268713 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:17:59.268633    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"]
Oct 30 19:17:59.273551 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:17:59.273517    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod972c42f1_2638_4521_9122_aa6ebad53891.slice/crio-ada8d3371be889ce2e858930a5d2f88e6d412bdd931e8d307b9cf2d4027c846f WatchSource:0}: Error finding container ada8d3371be889ce2e858930a5d2f88e6d412bdd931e8d307b9cf2d4027c846f: Status 404 returned error can't find the container with id ada8d3371be889ce2e858930a5d2f88e6d412bdd931e8d307b9cf2d4027c846f
Oct 30 19:18:00.256408 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:00.256376    4925 generic.go:356] "Generic (PLEG): container finished" podID="972c42f1-2638-4521-9122-aa6ebad53891" containerID="258e44f22ace8d4857d1a92296c50ea3841eb80a90818ed5c892ada1e294b7e2" exitCode=0
Oct 30 19:18:00.256578 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:00.256425    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls" event={"ID":"972c42f1-2638-4521-9122-aa6ebad53891","Type":"ContainerDied","Data":"258e44f22ace8d4857d1a92296c50ea3841eb80a90818ed5c892ada1e294b7e2"}
Oct 30 19:18:00.256578 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:00.256496    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls" event={"ID":"972c42f1-2638-4521-9122-aa6ebad53891","Type":"ContainerStarted","Data":"ada8d3371be889ce2e858930a5d2f88e6d412bdd931e8d307b9cf2d4027c846f"}
Oct 30 19:18:01.396159 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:01.396136    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:18:01.579235 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:01.579140    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-openshift-ca\") pod \"972c42f1-2638-4521-9122-aa6ebad53891\" (UID: \"972c42f1-2638-4521-9122-aa6ebad53891\") "
Oct 30 19:18:01.579235 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:01.579201    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/972c42f1-2638-4521-9122-aa6ebad53891-kube-api-access-ghkkl\" (UniqueName: \"kubernetes.io/projected/972c42f1-2638-4521-9122-aa6ebad53891-kube-api-access-ghkkl\") pod \"972c42f1-2638-4521-9122-aa6ebad53891\" (UID: \"972c42f1-2638-4521-9122-aa6ebad53891\") "
Oct 30 19:18:01.579487 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:01.579235    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-xks-ca\") pod \"972c42f1-2638-4521-9122-aa6ebad53891\" (UID: \"972c42f1-2638-4521-9122-aa6ebad53891\") "
Oct 30 19:18:01.579563 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:01.579491    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-openshift-ca" pod "972c42f1-2638-4521-9122-aa6ebad53891" (UID: "972c42f1-2638-4521-9122-aa6ebad53891"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:01.579625 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:01.579593    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-openshift-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:01.581543 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:01.581520    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-xks-ca" pod "972c42f1-2638-4521-9122-aa6ebad53891" (UID: "972c42f1-2638-4521-9122-aa6ebad53891"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:18:01.581657 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:01.581552    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/972c42f1-2638-4521-9122-aa6ebad53891-kube-api-access-ghkkl" pod "972c42f1-2638-4521-9122-aa6ebad53891" (UID: "972c42f1-2638-4521-9122-aa6ebad53891"). InnerVolumeSpecName "kube-api-access-ghkkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:01.680919 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:01.680891    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghkkl\" (UniqueName: \"kubernetes.io/projected/972c42f1-2638-4521-9122-aa6ebad53891-kube-api-access-ghkkl\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:01.680919 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:01.680917    4925 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/972c42f1-2638-4521-9122-aa6ebad53891-maas-api-xks-ca\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:02.268256 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:02.268219    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls" event={"ID":"972c42f1-2638-4521-9122-aa6ebad53891","Type":"ContainerDied","Data":"ada8d3371be889ce2e858930a5d2f88e6d412bdd931e8d307b9cf2d4027c846f"}
Oct 30 19:18:02.268256 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:02.268251    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3190c194-w59ls"
Oct 30 19:18:02.268639 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:02.268271    4925 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ada8d3371be889ce2e858930a5d2f88e6d412bdd931e8d307b9cf2d4027c846f"
Oct 30 19:18:02.989483 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:02.989441    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6" podUID="561dc359-e6c2-4519-b21e-687dffcd3a81" containerName="payload-processing" containerID="cri-o://88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc" gracePeriod=25
Oct 30 19:18:07.695857 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:07.695817    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main" containerID="cri-o://75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866" gracePeriod=45
Oct 30 19:18:07.946620 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:07.946570    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 19:18:08.028453 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.028413    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-home\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-home\") pod \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") "
Oct 30 19:18:08.028620 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.028486    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-dshm\") pod \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") "
Oct 30 19:18:08.028620 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.028533    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a51b10af-2839-4268-b3dc-d16dbcd5cd13-kube-api-access-8262m\" (UniqueName: \"kubernetes.io/projected/a51b10af-2839-4268-b3dc-d16dbcd5cd13-kube-api-access-8262m\") pod \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") "
Oct 30 19:18:08.028620 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.028599    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-model-cache\") pod \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") "
Oct 30 19:18:08.028780 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.028710    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-home" pod "a51b10af-2839-4268-b3dc-d16dbcd5cd13" (UID: "a51b10af-2839-4268-b3dc-d16dbcd5cd13"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:08.028780 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.028745    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tls-certs\" (UniqueName: \"kubernetes.io/secret/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tls-certs\") pod \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") "
Oct 30 19:18:08.028903 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.028799    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tmp-dir\") pod \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\" (UID: \"a51b10af-2839-4268-b3dc-d16dbcd5cd13\") "
Oct 30 19:18:08.028963 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.028942    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-model-cache" pod "a51b10af-2839-4268-b3dc-d16dbcd5cd13" (UID: "a51b10af-2839-4268-b3dc-d16dbcd5cd13"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:08.029147 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.029120    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tmp-dir" pod "a51b10af-2839-4268-b3dc-d16dbcd5cd13" (UID: "a51b10af-2839-4268-b3dc-d16dbcd5cd13"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:08.029251 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.029220    4925 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-home\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:08.029251 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.029243    4925 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-model-cache\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:08.030884 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.030857    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-dshm" pod "a51b10af-2839-4268-b3dc-d16dbcd5cd13" (UID: "a51b10af-2839-4268-b3dc-d16dbcd5cd13"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:08.030992 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.030940    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a51b10af-2839-4268-b3dc-d16dbcd5cd13-kube-api-access-8262m" pod "a51b10af-2839-4268-b3dc-d16dbcd5cd13" (UID: "a51b10af-2839-4268-b3dc-d16dbcd5cd13"). InnerVolumeSpecName "kube-api-access-8262m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:08.031444 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.031419    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tls-certs" pod "a51b10af-2839-4268-b3dc-d16dbcd5cd13" (UID: "a51b10af-2839-4268-b3dc-d16dbcd5cd13"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:18:08.130152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.130123    4925 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tls-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:08.130152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.130149    4925 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-tmp-dir\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:08.130152 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.130159    4925 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a51b10af-2839-4268-b3dc-d16dbcd5cd13-dshm\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:08.130397 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.130170    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8262m\" (UniqueName: \"kubernetes.io/projected/a51b10af-2839-4268-b3dc-d16dbcd5cd13-kube-api-access-8262m\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:08.300846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.300754    4925 generic.go:356] "Generic (PLEG): container finished" podID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerID="75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866" exitCode=0
Oct 30 19:18:08.300846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.300799    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" event={"ID":"a51b10af-2839-4268-b3dc-d16dbcd5cd13","Type":"ContainerDied","Data":"75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866"}
Oct 30 19:18:08.300846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.300801    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"
Oct 30 19:18:08.300846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.300831    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d" event={"ID":"a51b10af-2839-4268-b3dc-d16dbcd5cd13","Type":"ContainerDied","Data":"ea6f0ebfe29ae411576b8a529387bd4e5d621b080f3eb0eac8497815f5f25c58"}
Oct 30 19:18:08.300846 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.300850    4925 scope.go:122] "RemoveContainer" containerID="75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866"
Oct 30 19:18:08.310919 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.310901    4925 scope.go:122] "RemoveContainer" containerID="75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866"
Oct 30 19:18:08.311225 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:18:08.311202    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866\": container with ID starting with 75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866 not found: ID does not exist" containerID="75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866"
Oct 30 19:18:08.311317 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.311233    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866"} err="failed to get container status \"75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866\": rpc error: code = NotFound desc = could not find container \"75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866\": container with ID starting with 75190be88420753caf22e7fc7ba3c9641c3e9303e6d8687643fcf026a5dc2866 not found: ID does not exist"
Oct 30 19:18:08.324915 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.324893    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"]
Oct 30 19:18:08.329178 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.329156    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-85cb41/distinct-2-w1-85cb41-kserve-7698b5f77d-dj82d"]
Oct 30 19:18:08.422054 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:08.422024    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" path="/var/lib/kubelet/pods/a51b10af-2839-4268-b3dc-d16dbcd5cd13/volumes"
Oct 30 19:18:11.500516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:11.500482    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:12.920834 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:12.920784    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6" podUID="561dc359-e6c2-4519-b21e-687dffcd3a81" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:18:13.302280 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:18:13.302193    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:18:13.824815 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:13.824777    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:14.014504 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:14.014464    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:14.538722 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:14.538681    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:20.481547 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:20.481502    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:20.684574 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:20.684530    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:20.720785 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:20.720748    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"]
Oct 30 19:18:20.721023 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:20.720994    4925 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7" podUID="7d22fee8-13c0-49d8-8f98-c107c33834e6" containerName="istio-proxy" containerID="cri-o://7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43" gracePeriod=30
Oct 30 19:18:20.873516 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:20.873425    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:21.611055 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:21.611020    4925 patch_prober.go:29] interesting pod/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:18:21.611481 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:21.611083    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7" podUID="7d22fee8-13c0-49d8-8f98-c107c33834e6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:18:22.920777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:22.920741    4925 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6" podUID="561dc359-e6c2-4519-b21e-687dffcd3a81" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:18:23.262237 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.262197    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"
Oct 30 19:18:23.378006 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.377967    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/561dc359-e6c2-4519-b21e-687dffcd3a81-kube-api-access-xr7qp\" (UniqueName: \"kubernetes.io/projected/561dc359-e6c2-4519-b21e-687dffcd3a81-kube-api-access-xr7qp\") pod \"561dc359-e6c2-4519-b21e-687dffcd3a81\" (UID: \"561dc359-e6c2-4519-b21e-687dffcd3a81\") "
Oct 30 19:18:23.378264 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.378133    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/561dc359-e6c2-4519-b21e-687dffcd3a81-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/561dc359-e6c2-4519-b21e-687dffcd3a81-plugins-config-volume\") pod \"561dc359-e6c2-4519-b21e-687dffcd3a81\" (UID: \"561dc359-e6c2-4519-b21e-687dffcd3a81\") "
Oct 30 19:18:23.378629 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.378601    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/561dc359-e6c2-4519-b21e-687dffcd3a81-plugins-config-volume" pod "561dc359-e6c2-4519-b21e-687dffcd3a81" (UID: "561dc359-e6c2-4519-b21e-687dffcd3a81"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:23.381172 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.381146    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/561dc359-e6c2-4519-b21e-687dffcd3a81-kube-api-access-xr7qp" pod "561dc359-e6c2-4519-b21e-687dffcd3a81" (UID: "561dc359-e6c2-4519-b21e-687dffcd3a81"). InnerVolumeSpecName "kube-api-access-xr7qp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:23.394431 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.394405    4925 generic.go:356] "Generic (PLEG): container finished" podID="561dc359-e6c2-4519-b21e-687dffcd3a81" containerID="88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc" exitCode=0
Oct 30 19:18:23.394532 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.394455    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"
Oct 30 19:18:23.394532 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.394488    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6" event={"ID":"561dc359-e6c2-4519-b21e-687dffcd3a81","Type":"ContainerDied","Data":"88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc"}
Oct 30 19:18:23.394626 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.394568    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6" event={"ID":"561dc359-e6c2-4519-b21e-687dffcd3a81","Type":"ContainerDied","Data":"ae412e2dce3eb522ae091c32b4eae17bacbcfa310d2e5c715696ef00823c25d9"}
Oct 30 19:18:23.394626 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.394589    4925 scope.go:122] "RemoveContainer" containerID="88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc"
Oct 30 19:18:23.410353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.409293    4925 scope.go:122] "RemoveContainer" containerID="88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc"
Oct 30 19:18:23.410353 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:18:23.409738    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc\": container with ID starting with 88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc not found: ID does not exist" containerID="88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc"
Oct 30 19:18:23.410353 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.409776    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc"} err="failed to get container status \"88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc\": rpc error: code = NotFound desc = could not find container \"88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc\": container with ID starting with 88cda1f8b7d5de7386a477e51d0bf47f5245e374fa698636dff37429a7412ccc not found: ID does not exist"
Oct 30 19:18:23.426378 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.426352    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"]
Oct 30 19:18:23.432482 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.432457    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-85cb41-558557d48b-zm5n6"]
Oct 30 19:18:23.479377 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.479350    4925 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/561dc359-e6c2-4519-b21e-687dffcd3a81-plugins-config-volume\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.479503 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.479383    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr7qp\" (UniqueName: \"kubernetes.io/projected/561dc359-e6c2-4519-b21e-687dffcd3a81-kube-api-access-xr7qp\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.565310 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:23.565234    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-hg8tr_43fb2f5e-93e5-4aef-8f44-1aefbc65d567/manager/0.log"
Oct 30 19:18:24.421540 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:24.421507    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="561dc359-e6c2-4519-b21e-687dffcd3a81" path="/var/lib/kubelet/pods/561dc359-e6c2-4519-b21e-687dffcd3a81/volumes"
Oct 30 19:18:24.718051 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:24.717967    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6c45775ff9-c2ghb_c536d86d-2397-47bf-a44f-47b0bf40dd02/manager/0.log"
Oct 30 19:18:25.915766 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:25.915728    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8486858879-ttlr5_e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9/manager/0.log"
Oct 30 19:18:26.396375 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.396350    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 19:18:26.411194 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.411166    4925 generic.go:356] "Generic (PLEG): container finished" podID="7d22fee8-13c0-49d8-8f98-c107c33834e6" containerID="7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43" exitCode=0
Oct 30 19:18:26.411304 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.411214    4925 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"
Oct 30 19:18:26.411304 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.411214    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7" event={"ID":"7d22fee8-13c0-49d8-8f98-c107c33834e6","Type":"ContainerDied","Data":"7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43"}
Oct 30 19:18:26.411388 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.411347    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7" event={"ID":"7d22fee8-13c0-49d8-8f98-c107c33834e6","Type":"ContainerDied","Data":"452370e86070ca9391ad5deddfa7a767df6ea4e77ea5e462fd21d66fff38332a"}
Oct 30 19:18:26.411388 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.411381    4925 scope.go:122] "RemoveContainer" containerID="7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43"
Oct 30 19:18:26.420872 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.420847    4925 scope.go:122] "RemoveContainer" containerID="7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43"
Oct 30 19:18:26.423396 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:18:26.423375    4925 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43\": container with ID starting with 7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43 not found: ID does not exist" containerID="7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43"
Oct 30 19:18:26.423509 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.423409    4925 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43"} err="failed to get container status \"7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43\": rpc error: code = NotFound desc = could not find container \"7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43\": container with ID starting with 7da5138c9c75f382d0f851e23b5d342164acc9b536bae0f4958a9ff28d0f1d43 not found: ID does not exist"
Oct 30 19:18:26.501264 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501220    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-certs\") pod \"7d22fee8-13c0-49d8-8f98-c107c33834e6\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") "
Oct 30 19:18:26.501369 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501273    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-token\" (UniqueName: \"kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-token\") pod \"7d22fee8-13c0-49d8-8f98-c107c33834e6\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") "
Oct 30 19:18:26.501369 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501297    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-podinfo\") pod \"7d22fee8-13c0-49d8-8f98-c107c33834e6\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") "
Oct 30 19:18:26.501369 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501324    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d22fee8-13c0-49d8-8f98-c107c33834e6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d22fee8-13c0-49d8-8f98-c107c33834e6-istiod-ca-cert\") pod \"7d22fee8-13c0-49d8-8f98-c107c33834e6\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") "
Oct 30 19:18:26.501369 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501342    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-socket\") pod \"7d22fee8-13c0-49d8-8f98-c107c33834e6\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") "
Oct 30 19:18:26.501369 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501361    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-data\") pod \"7d22fee8-13c0-49d8-8f98-c107c33834e6\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") "
Oct 30 19:18:26.501650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501383    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-envoy\") pod \"7d22fee8-13c0-49d8-8f98-c107c33834e6\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") "
Oct 30 19:18:26.501650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501413    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-credential-socket\") pod \"7d22fee8-13c0-49d8-8f98-c107c33834e6\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") "
Oct 30 19:18:26.501650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501493    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-certs" pod "7d22fee8-13c0-49d8-8f98-c107c33834e6" (UID: "7d22fee8-13c0-49d8-8f98-c107c33834e6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:26.501650 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501517    4925 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-kube-api-access-x5lwp\" (UniqueName: \"kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-kube-api-access-x5lwp\") pod \"7d22fee8-13c0-49d8-8f98-c107c33834e6\" (UID: \"7d22fee8-13c0-49d8-8f98-c107c33834e6\") "
Oct 30 19:18:26.501869 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501671    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-socket" pod "7d22fee8-13c0-49d8-8f98-c107c33834e6" (UID: "7d22fee8-13c0-49d8-8f98-c107c33834e6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:26.501869 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501685    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-data" pod "7d22fee8-13c0-49d8-8f98-c107c33834e6" (UID: "7d22fee8-13c0-49d8-8f98-c107c33834e6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:26.501978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501820    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d22fee8-13c0-49d8-8f98-c107c33834e6-istiod-ca-cert" pod "7d22fee8-13c0-49d8-8f98-c107c33834e6" (UID: "7d22fee8-13c0-49d8-8f98-c107c33834e6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:26.501978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501906    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-credential-socket" pod "7d22fee8-13c0-49d8-8f98-c107c33834e6" (UID: "7d22fee8-13c0-49d8-8f98-c107c33834e6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:26.501978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501886    4925 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-certs\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:26.501978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501950    4925 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-workload-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:26.501978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.501969    4925 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-data\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:26.503691 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.503668    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-token" pod "7d22fee8-13c0-49d8-8f98-c107c33834e6" (UID: "7d22fee8-13c0-49d8-8f98-c107c33834e6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:26.503776 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.503701    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-envoy" pod "7d22fee8-13c0-49d8-8f98-c107c33834e6" (UID: "7d22fee8-13c0-49d8-8f98-c107c33834e6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:26.503776 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.503722    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-podinfo" pod "7d22fee8-13c0-49d8-8f98-c107c33834e6" (UID: "7d22fee8-13c0-49d8-8f98-c107c33834e6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:18:26.503858 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.503808    4925 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-kube-api-access-x5lwp" pod "7d22fee8-13c0-49d8-8f98-c107c33834e6" (UID: "7d22fee8-13c0-49d8-8f98-c107c33834e6"). InnerVolumeSpecName "kube-api-access-x5lwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:26.546281 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.546262    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-gx2x5_7242bc65-404e-481b-b596-968eb61f0f1c/proxy/0.log"
Oct 30 19:18:26.603157 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.603136    4925 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5lwp\" (UniqueName: \"kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-kube-api-access-x5lwp\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:26.603157 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.603159    4925 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-token\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:26.603307 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.603168    4925 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-podinfo\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:26.603307 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.603178    4925 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d22fee8-13c0-49d8-8f98-c107c33834e6-istiod-ca-cert\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:26.603307 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.603186    4925 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-istio-envoy\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:26.603307 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.603194    4925 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d22fee8-13c0-49d8-8f98-c107c33834e6-credential-socket\") on node \"ip-10-0-134-58.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:26.734929 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.734906    4925 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"]
Oct 30 19:18:26.739777 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:26.739754    4925 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-85cb41-openshift-default-74bfbc4465-ldpr7"]
Oct 30 19:18:28.423245 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:28.423201    4925 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d22fee8-13c0-49d8-8f98-c107c33834e6" path="/var/lib/kubelet/pods/7d22fee8-13c0-49d8-8f98-c107c33834e6/volumes"
Oct 30 19:18:29.348343 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:29.348307    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:30.806863 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:30.806824    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:36.413033 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:36.412983    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:37.730610 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:37.730574    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6c45775ff9-c2ghb_c536d86d-2397-47bf-a44f-47b0bf40dd02/manager/0.log"
Oct 30 19:18:38.935928 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:38.935895    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8486858879-ttlr5_e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9/manager/0.log"
Oct 30 19:18:39.512061 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:39.512033    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-gx2x5_7242bc65-404e-481b-b596-968eb61f0f1c/proxy/0.log"
Oct 30 19:18:40.248445 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:40.248410    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-76ff54bc44-n2nzz_a7c8e0d4-23c8-407b-b624-c84b8f2f31c6/maas-api/0.log"
Oct 30 19:18:41.174002 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:41.173971    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3190c194-w59ls_972c42f1-2638-4521-9122-aa6ebad53891/revoke-keys/0.log"
Oct 30 19:18:41.528174 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:41.528062    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-7f6346b4-9wf45_d2eed241-cb88-48e4-9cba-38e63d153cc3/revoke-keys/0.log"
Oct 30 19:18:43.300279 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:18:43.300244    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:18:45.691704 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:45.691674    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-hg8tr_43fb2f5e-93e5-4aef-8f44-1aefbc65d567/manager/0.log"
Oct 30 19:18:47.437605 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:47.437567    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-rqh7r_ad2e18b3-d6e4-468b-8f40-6e418841774e/discovery/0.log"
Oct 30 19:18:47.820618 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:47.820521    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f89dc687f-bwx2f_24691fdd-6f67-43e4-bd56-446e31b2c0a2/kube-auth-proxy/0.log"
Oct 30 19:18:48.175820 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:48.175788    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-68cb9b4b95-78fkg_63ba4746-17cf-46a6-8517-720d4fbdb75a/istio-proxy/0.log"
Oct 30 19:18:48.519804 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:48.519720    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-68cb9b4b95-p2rlg_90d0440c-184a-473e-897a-8fe6311a2ddf/istio-proxy/0.log"
Oct 30 19:18:49.420462 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:49.420428    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-6hs9t_6edaba68-471d-475d-9a07-16ecd7bd4e8e/payload-pre-processing/0.log"
Oct 30 19:18:49.592158 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:49.592129    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-vfsrt_77704542-8e8e-4d3e-a4bd-e07d4f0e79a0/payload-processing/0.log"
Oct 30 19:18:49.778914 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:49.778846    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59c4cc76f-fbdqx_2c77e172-a80d-4eda-986d-ec8c3c52e131/router/0.log"
Oct 30 19:18:50.735808 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:50.735781    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-kpwlj_6d32df70-08f9-494a-81d5-868f1511dcb4/main/0.log"
Oct 30 19:18:51.083628 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:51.083548    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-68f2m_b7545f38-34b7-4c0d-9887-1a065c2231b0/main/0.log"
Oct 30 19:18:51.259494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:51.259455    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-npxnt_af327665-1858-4fc9-a89a-772751ad7921/main/0.log"
Oct 30 19:18:51.619472 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:18:51.619437    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-qht75_bd42a605-1c88-4212-9199-1f3572a08984/main/0.log"
Oct 30 19:19:03.499532 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.499496    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jfcsw/must-gather-bw2zm"]
Oct 30 19:19:03.502039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.499961    4925 cpu_manager.go:414] "containerMap: removing container" podUID="561dc359-e6c2-4519-b21e-687dffcd3a81" containerName="payload-processing"
Oct 30 19:19:03.502039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.499977    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="561dc359-e6c2-4519-b21e-687dffcd3a81" containerName="payload-processing"
Oct 30 19:19:03.502039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.499996    4925 cpu_manager.go:414] "containerMap: removing container" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main"
Oct 30 19:19:03.502039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.500002    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main"
Oct 30 19:19:03.502039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.500013    4925 cpu_manager.go:414] "containerMap: removing container" podUID="7d22fee8-13c0-49d8-8f98-c107c33834e6" containerName="istio-proxy"
Oct 30 19:19:03.502039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.500020    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d22fee8-13c0-49d8-8f98-c107c33834e6" containerName="istio-proxy"
Oct 30 19:19:03.502039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.500028    4925 cpu_manager.go:414] "containerMap: removing container" podUID="972c42f1-2638-4521-9122-aa6ebad53891" containerName="revoke-keys"
Oct 30 19:19:03.502039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.500033    4925 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="972c42f1-2638-4521-9122-aa6ebad53891" containerName="revoke-keys"
Oct 30 19:19:03.502039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.500142    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="972c42f1-2638-4521-9122-aa6ebad53891" containerName="revoke-keys"
Oct 30 19:19:03.502039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.500165    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="561dc359-e6c2-4519-b21e-687dffcd3a81" containerName="payload-processing"
Oct 30 19:19:03.502039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.500175    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="a51b10af-2839-4268-b3dc-d16dbcd5cd13" containerName="main"
Oct 30 19:19:03.502039 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.500184    4925 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d22fee8-13c0-49d8-8f98-c107c33834e6" containerName="istio-proxy"
Oct 30 19:19:03.503338 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.503323    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jfcsw/must-gather-bw2zm"
Oct 30 19:19:03.538638 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.538613    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jfcsw/must-gather-bw2zm"]
Oct 30 19:19:03.554156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.554129    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jfcsw\"/\"default-dockercfg-pxnx9\""
Oct 30 19:19:03.554617 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.554601    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jfcsw\"/\"openshift-service-ca.crt\""
Oct 30 19:19:03.554617 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.554611    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jfcsw\"/\"kube-root-ca.crt\""
Oct 30 19:19:03.593190 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.593159    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9v8ng/must-gather-sn8rt"]
Oct 30 19:19:03.596858 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.596836    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9v8ng/must-gather-sn8rt"
Oct 30 19:19:03.605397 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.605376    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-47xdf/must-gather-88g2c"]
Oct 30 19:19:03.609450 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.609432    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47xdf/must-gather-88g2c"
Oct 30 19:19:03.617221 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.617194    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:03.617356 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.617308    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9v8ng/must-gather-sn8rt"]
Oct 30 19:19:03.623231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.623207    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9v8ng\"/\"kube-root-ca.crt\""
Oct 30 19:19:03.623231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.623228    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-47xdf\"/\"openshift-service-ca.crt\""
Oct 30 19:19:03.623407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.623229    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-47xdf\"/\"kube-root-ca.crt\""
Oct 30 19:19:03.623407 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.623376    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9v8ng\"/\"default-dockercfg-r697p\""
Oct 30 19:19:03.624222 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.624200    4925 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9v8ng\"/\"openshift-service-ca.crt\""
Oct 30 19:19:03.624334 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.624297    4925 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-47xdf\"/\"default-dockercfg-xpwfn\""
Oct 30 19:19:03.626615 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.626588    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b792n\" (UniqueName: \"kubernetes.io/projected/709512e2-67af-47cd-8f5e-e8ddb7431f63-kube-api-access-b792n\") pod \"must-gather-bw2zm\" (UID: \"709512e2-67af-47cd-8f5e-e8ddb7431f63\") " pod="openshift-must-gather-jfcsw/must-gather-bw2zm"
Oct 30 19:19:03.626733 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.626686    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4c04215d-579a-46f8-8a59-bbfda01500ff-must-gather-output\") pod \"must-gather-sn8rt\" (UID: \"4c04215d-579a-46f8-8a59-bbfda01500ff\") " pod="openshift-must-gather-9v8ng/must-gather-sn8rt"
Oct 30 19:19:03.626832 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.626809    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nnhm\" (UniqueName: \"kubernetes.io/projected/4c04215d-579a-46f8-8a59-bbfda01500ff-kube-api-access-5nnhm\") pod \"must-gather-sn8rt\" (UID: \"4c04215d-579a-46f8-8a59-bbfda01500ff\") " pod="openshift-must-gather-9v8ng/must-gather-sn8rt"
Oct 30 19:19:03.626898 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.626872    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/709512e2-67af-47cd-8f5e-e8ddb7431f63-must-gather-output\") pod \"must-gather-bw2zm\" (UID: \"709512e2-67af-47cd-8f5e-e8ddb7431f63\") " pod="openshift-must-gather-jfcsw/must-gather-bw2zm"
Oct 30 19:19:03.647775 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.647750    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47xdf/must-gather-88g2c"]
Oct 30 19:19:03.728289 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.728253    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d1dff971-ab3c-4c34-8781-4921c1e7bcf6-must-gather-output\") pod \"must-gather-88g2c\" (UID: \"d1dff971-ab3c-4c34-8781-4921c1e7bcf6\") " pod="openshift-must-gather-47xdf/must-gather-88g2c"
Oct 30 19:19:03.728506 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.728309    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b792n\" (UniqueName: \"kubernetes.io/projected/709512e2-67af-47cd-8f5e-e8ddb7431f63-kube-api-access-b792n\") pod \"must-gather-bw2zm\" (UID: \"709512e2-67af-47cd-8f5e-e8ddb7431f63\") " pod="openshift-must-gather-jfcsw/must-gather-bw2zm"
Oct 30 19:19:03.728506 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.728397    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4c04215d-579a-46f8-8a59-bbfda01500ff-must-gather-output\") pod \"must-gather-sn8rt\" (UID: \"4c04215d-579a-46f8-8a59-bbfda01500ff\") " pod="openshift-must-gather-9v8ng/must-gather-sn8rt"
Oct 30 19:19:03.728506 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.728442    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnj2s\" (UniqueName: \"kubernetes.io/projected/d1dff971-ab3c-4c34-8781-4921c1e7bcf6-kube-api-access-bnj2s\") pod \"must-gather-88g2c\" (UID: \"d1dff971-ab3c-4c34-8781-4921c1e7bcf6\") " pod="openshift-must-gather-47xdf/must-gather-88g2c"
Oct 30 19:19:03.728695 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.728560    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nnhm\" (UniqueName: \"kubernetes.io/projected/4c04215d-579a-46f8-8a59-bbfda01500ff-kube-api-access-5nnhm\") pod \"must-gather-sn8rt\" (UID: \"4c04215d-579a-46f8-8a59-bbfda01500ff\") " pod="openshift-must-gather-9v8ng/must-gather-sn8rt"
Oct 30 19:19:03.728695 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.728621    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/709512e2-67af-47cd-8f5e-e8ddb7431f63-must-gather-output\") pod \"must-gather-bw2zm\" (UID: \"709512e2-67af-47cd-8f5e-e8ddb7431f63\") " pod="openshift-must-gather-jfcsw/must-gather-bw2zm"
Oct 30 19:19:03.728811 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.728748    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4c04215d-579a-46f8-8a59-bbfda01500ff-must-gather-output\") pod \"must-gather-sn8rt\" (UID: \"4c04215d-579a-46f8-8a59-bbfda01500ff\") " pod="openshift-must-gather-9v8ng/must-gather-sn8rt"
Oct 30 19:19:03.728910 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.728891    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/709512e2-67af-47cd-8f5e-e8ddb7431f63-must-gather-output\") pod \"must-gather-bw2zm\" (UID: \"709512e2-67af-47cd-8f5e-e8ddb7431f63\") " pod="openshift-must-gather-jfcsw/must-gather-bw2zm"
Oct 30 19:19:03.742560 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.742531    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nnhm\" (UniqueName: \"kubernetes.io/projected/4c04215d-579a-46f8-8a59-bbfda01500ff-kube-api-access-5nnhm\") pod \"must-gather-sn8rt\" (UID: \"4c04215d-579a-46f8-8a59-bbfda01500ff\") " pod="openshift-must-gather-9v8ng/must-gather-sn8rt"
Oct 30 19:19:03.744059 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.744039    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b792n\" (UniqueName: \"kubernetes.io/projected/709512e2-67af-47cd-8f5e-e8ddb7431f63-kube-api-access-b792n\") pod \"must-gather-bw2zm\" (UID: \"709512e2-67af-47cd-8f5e-e8ddb7431f63\") " pod="openshift-must-gather-jfcsw/must-gather-bw2zm"
Oct 30 19:19:03.829509 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.829434    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnj2s\" (UniqueName: \"kubernetes.io/projected/d1dff971-ab3c-4c34-8781-4921c1e7bcf6-kube-api-access-bnj2s\") pod \"must-gather-88g2c\" (UID: \"d1dff971-ab3c-4c34-8781-4921c1e7bcf6\") " pod="openshift-must-gather-47xdf/must-gather-88g2c"
Oct 30 19:19:03.829643 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.829529    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d1dff971-ab3c-4c34-8781-4921c1e7bcf6-must-gather-output\") pod \"must-gather-88g2c\" (UID: \"d1dff971-ab3c-4c34-8781-4921c1e7bcf6\") " pod="openshift-must-gather-47xdf/must-gather-88g2c"
Oct 30 19:19:03.829848 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.829832    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d1dff971-ab3c-4c34-8781-4921c1e7bcf6-must-gather-output\") pod \"must-gather-88g2c\" (UID: \"d1dff971-ab3c-4c34-8781-4921c1e7bcf6\") " pod="openshift-must-gather-47xdf/must-gather-88g2c"
Oct 30 19:19:03.837304 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.837279    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jfcsw/must-gather-bw2zm"
Oct 30 19:19:03.846005 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.845982    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnj2s\" (UniqueName: \"kubernetes.io/projected/d1dff971-ab3c-4c34-8781-4921c1e7bcf6-kube-api-access-bnj2s\") pod \"must-gather-88g2c\" (UID: \"d1dff971-ab3c-4c34-8781-4921c1e7bcf6\") " pod="openshift-must-gather-47xdf/must-gather-88g2c"
Oct 30 19:19:03.911620 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.911585    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9v8ng/must-gather-sn8rt"
Oct 30 19:19:03.912836 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.912800    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:03.934674 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:03.934641    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47xdf/must-gather-88g2c"
Oct 30 19:19:04.119090 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:04.119067    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jfcsw/must-gather-bw2zm"]
Oct 30 19:19:04.121142 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:19:04.121111    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod709512e2_67af_47cd_8f5e_e8ddb7431f63.slice/crio-b65ee3d8965dfc7a08211c182056cdc85d0a3a26a53b96fc34299d813cbceaf1 WatchSource:0}: Error finding container b65ee3d8965dfc7a08211c182056cdc85d0a3a26a53b96fc34299d813cbceaf1: Status 404 returned error can't find the container with id b65ee3d8965dfc7a08211c182056cdc85d0a3a26a53b96fc34299d813cbceaf1
Oct 30 19:19:04.122937 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:04.122915    4925 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:19:04.189634 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:04.189610    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9v8ng/must-gather-sn8rt"]
Oct 30 19:19:04.192155 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:19:04.192130    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c04215d_579a_46f8_8a59_bbfda01500ff.slice/crio-ae24ae3db1a205cfe76e40b103c6bc679a48735b714e9c8157f4d9b2993432ca WatchSource:0}: Error finding container ae24ae3db1a205cfe76e40b103c6bc679a48735b714e9c8157f4d9b2993432ca: Status 404 returned error can't find the container with id ae24ae3db1a205cfe76e40b103c6bc679a48735b714e9c8157f4d9b2993432ca
Oct 30 19:19:04.218082 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:04.218057    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47xdf/must-gather-88g2c"]
Oct 30 19:19:04.219450 ip-10-0-134-58 kubenswrapper[4925]: W1030 19:19:04.219425    4925 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1dff971_ab3c_4c34_8781_4921c1e7bcf6.slice/crio-82c00716b7cae66d61bf6b4937e44d1ca402d8ef11d008452386c87dbec73162 WatchSource:0}: Error finding container 82c00716b7cae66d61bf6b4937e44d1ca402d8ef11d008452386c87dbec73162: Status 404 returned error can't find the container with id 82c00716b7cae66d61bf6b4937e44d1ca402d8ef11d008452386c87dbec73162
Oct 30 19:19:04.607422 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:04.607385    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47xdf/must-gather-88g2c" event={"ID":"d1dff971-ab3c-4c34-8781-4921c1e7bcf6","Type":"ContainerStarted","Data":"82c00716b7cae66d61bf6b4937e44d1ca402d8ef11d008452386c87dbec73162"}
Oct 30 19:19:04.608566 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:04.608534    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jfcsw/must-gather-bw2zm" event={"ID":"709512e2-67af-47cd-8f5e-e8ddb7431f63","Type":"ContainerStarted","Data":"b65ee3d8965dfc7a08211c182056cdc85d0a3a26a53b96fc34299d813cbceaf1"}
Oct 30 19:19:04.609770 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:04.609748    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9v8ng/must-gather-sn8rt" event={"ID":"4c04215d-579a-46f8-8a59-bbfda01500ff","Type":"ContainerStarted","Data":"ae24ae3db1a205cfe76e40b103c6bc679a48735b714e9c8157f4d9b2993432ca"}
Oct 30 19:19:05.618614 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:05.618580    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47xdf/must-gather-88g2c" event={"ID":"d1dff971-ab3c-4c34-8781-4921c1e7bcf6","Type":"ContainerStarted","Data":"58fbe810eeac5fc6a1409dc6ae83047a8c21316f296f27367063bd5dfcabe03d"}
Oct 30 19:19:05.619070 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:05.618623    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47xdf/must-gather-88g2c" event={"ID":"d1dff971-ab3c-4c34-8781-4921c1e7bcf6","Type":"ContainerStarted","Data":"61d71adfadf4c7310f8393c775bc5e7f81a50d66f683284013912a457b98783d"}
Oct 30 19:19:05.637977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:05.637901    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-47xdf/must-gather-88g2c" podStartSLOduration=1.7197841980000002 podStartE2EDuration="2.637881546s" totalImagesPullingTime="918.097348ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:05.635155403 +0000 UTC m=+2567.351500479" watchObservedRunningTime="2026-10-30 19:19:05.637881546 +0000 UTC m=+2567.354226620"
Oct 30 19:19:07.715241 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:07.715160    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cmf2j_ac85734e-3f97-473f-b3ab-8aadd898f381/global-pull-secret-syncer/0.log"
Oct 30 19:19:07.807231 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:07.807201    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-985h5_b0939980-01bb-4ae3-9c95-5e190b719cbe/connection-checker/0.log"
Oct 30 19:19:07.850534 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:07.850498    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-tfwdw_ae2cac8d-509e-4e22-b35c-ab6d662fdf24/connection-checker/0.log"
Oct 30 19:19:07.881876 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:07.881844    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-v788b_f16c549e-41e6-4721-ab5f-91b37f05a97c/connection-checker/0.log"
Oct 30 19:19:07.990572 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:07.990470    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log"
Oct 30 19:19:08.158886 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:08.158849    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-58.us-east-2.compute.internal_749244aef4ca28f482f48e6f6d30c5f8/haproxy/0.log"
Oct 30 19:19:08.593548 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:08.593504    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:11.662433 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:11.661907    4925 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9pnlw_3762f795-944b-458e-9458-c26eaf133871/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:13.396786 ip-10-0-134-58 kubenswrapper[4925]: E1030 19:19:13.396742    4925 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9czjj" containerName="extractor"
Oct 30 19:19:16.983552 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:16.983518    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-hg8tr_43fb2f5e-93e5-4aef-8f44-1aefbc65d567/manager/0.log"
Oct 30 19:19:18.854955 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:18.854913    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-n92pf_f0ba138d-0a24-4f77-b1f4-c0861a48797f/cluster-monitoring-operator/0.log"
Oct 30 19:19:19.027215 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:19.027142    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bdfjd_1c47621d-ebe5-4469-a944-c74fd68df26f/node-exporter/0.log"
Oct 30 19:19:19.050154 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:19.050076    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bdfjd_1c47621d-ebe5-4469-a944-c74fd68df26f/kube-rbac-proxy/0.log"
Oct 30 19:19:19.074503 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:19.074471    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bdfjd_1c47621d-ebe5-4469-a944-c74fd68df26f/init-textfile/0.log"
Oct 30 19:19:19.926057 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:19.926017    4925 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"]
Oct 30 19:19:19.942594 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:19.942550    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"]
Oct 30 19:19:19.942783 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:19.942692    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.089479 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.089424    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bctpt\" (UniqueName: \"kubernetes.io/projected/4a695077-d88f-49a8-b866-f94e52a30ea7-kube-api-access-bctpt\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.089675 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.089508    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4a695077-d88f-49a8-b866-f94e52a30ea7-podres\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.089675 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.089637    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4a695077-d88f-49a8-b866-f94e52a30ea7-proc\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.089762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.089687    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a695077-d88f-49a8-b866-f94e52a30ea7-sys\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.089762 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.089714    4925 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4a695077-d88f-49a8-b866-f94e52a30ea7-lib-modules\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.191332 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.191233    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a695077-d88f-49a8-b866-f94e52a30ea7-sys\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.191332 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.191295    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4a695077-d88f-49a8-b866-f94e52a30ea7-lib-modules\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.191581 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.191355    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bctpt\" (UniqueName: \"kubernetes.io/projected/4a695077-d88f-49a8-b866-f94e52a30ea7-kube-api-access-bctpt\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.191581 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.191396    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4a695077-d88f-49a8-b866-f94e52a30ea7-podres\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.191581 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.191526    4925 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4a695077-d88f-49a8-b866-f94e52a30ea7-proc\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.191768 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.191638    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4a695077-d88f-49a8-b866-f94e52a30ea7-proc\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.191768 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.191700    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4a695077-d88f-49a8-b866-f94e52a30ea7-sys\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.191768 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.191764    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4a695077-d88f-49a8-b866-f94e52a30ea7-lib-modules\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.192139 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.192118    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4a695077-d88f-49a8-b866-f94e52a30ea7-podres\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.201336 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.201300    4925 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bctpt\" (UniqueName: \"kubernetes.io/projected/4a695077-d88f-49a8-b866-f94e52a30ea7-kube-api-access-bctpt\") pod \"perf-node-gather-daemonset-zz652\" (UID: \"4a695077-d88f-49a8-b866-f94e52a30ea7\") " pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.268574 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.268533    4925 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.578501 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.575668    4925 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"]
Oct 30 19:19:20.919274 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.919236    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652" event={"ID":"4a695077-d88f-49a8-b866-f94e52a30ea7","Type":"ContainerStarted","Data":"a6e681c6c31c982f749e7167ffc1f24868740c6bc238e04496a3bd23986c96a6"}
Oct 30 19:19:20.919494 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.919303    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652" event={"ID":"4a695077-d88f-49a8-b866-f94e52a30ea7","Type":"ContainerStarted","Data":"ed9886714d5ae200f588eb4131b97cc073158c1263812a18c209b1e7f53aa616"}
Oct 30 19:19:20.919576 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.919499    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:20.936752 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:20.936687    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652" podStartSLOduration=1.936667721 podStartE2EDuration="1.936667721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:20.934900599 +0000 UTC m=+2582.651245751" watchObservedRunningTime="2026-10-30 19:19:20.936667721 +0000 UTC m=+2582.653012798"
Oct 30 19:19:21.412012 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:21.411972    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-k9dgt_f478aac2-de91-4b13-836b-ba09809222f0/networking-console-plugin/0.log"
Oct 30 19:19:22.241635 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:22.241574    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/2.log"
Oct 30 19:19:22.256499 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:22.256354    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-68flm_5cf855c4-4e7e-4669-9adc-583e22e27e73/console-operator/3.log"
Oct 30 19:19:22.952982 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:22.952909    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jfcsw/must-gather-bw2zm" event={"ID":"709512e2-67af-47cd-8f5e-e8ddb7431f63","Type":"ContainerStarted","Data":"d0fb1e901129a79e1f4ed2719f0a156af19953d819cef08c9880aa7ba3f39469"}
Oct 30 19:19:23.895133 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:23.892440    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-ljxjz_4cd65688-51e1-4b8c-8dc5-6e3771a74e66/volume-data-source-validator/0.log"
Oct 30 19:19:23.971997 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:23.971950    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9v8ng/must-gather-sn8rt" event={"ID":"4c04215d-579a-46f8-8a59-bbfda01500ff","Type":"ContainerStarted","Data":"1581bfaabc0866c5ba1aeff5aeb0ae4fa2c6a37390445da4b34d95315c4152e9"}
Oct 30 19:19:23.971997 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:23.972005    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9v8ng/must-gather-sn8rt" event={"ID":"4c04215d-579a-46f8-8a59-bbfda01500ff","Type":"ContainerStarted","Data":"ad4e0381a444323068d02df4e3a6127cd70cd036be3fd0a887b2ac06208f794b"}
Oct 30 19:19:23.975021 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:23.974990    4925 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jfcsw/must-gather-bw2zm" event={"ID":"709512e2-67af-47cd-8f5e-e8ddb7431f63","Type":"ContainerStarted","Data":"854b94ea1f579718b45d41a5a23681c2156e9f48add5d57617b74b664d160ea2"}
Oct 30 19:19:23.993638 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:23.993190    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-9v8ng/must-gather-sn8rt" podStartSLOduration=2.039566567 podStartE2EDuration="20.993172418s" totalImagesPullingTime="18.953605851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:23.991664228 +0000 UTC m=+2585.708009307" watchObservedRunningTime="2026-10-30 19:19:23.993172418 +0000 UTC m=+2585.709517495"
Oct 30 19:19:24.021402 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:24.019478    4925 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-jfcsw/must-gather-bw2zm" podStartSLOduration=2.379765358 podStartE2EDuration="21.019459055s" totalImagesPullingTime="18.639693697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:24.015420997 +0000 UTC m=+2585.731766146" watchObservedRunningTime="2026-10-30 19:19:24.019459055 +0000 UTC m=+2585.735804133"
Oct 30 19:19:24.828766 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:24.828730    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6c45775ff9-c2ghb_c536d86d-2397-47bf-a44f-47b0bf40dd02/manager/0.log"
Oct 30 19:19:25.112814 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:25.112357    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6c45775ff9-c2ghb_c536d86d-2397-47bf-a44f-47b0bf40dd02/manager/0.log"
Oct 30 19:19:26.232132 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:26.229826    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8486858879-ttlr5_e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9/manager/0.log"
Oct 30 19:19:26.336886 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:26.336789    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jcn7p_734db718-c901-4fe7-a8e1-b90823ccd66c/dns/0.log"
Oct 30 19:19:26.367947 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:26.367908    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jcn7p_734db718-c901-4fe7-a8e1-b90823ccd66c/kube-rbac-proxy/0.log"
Oct 30 19:19:26.475977 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:26.475803    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8486858879-ttlr5_e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9/manager/0.log"
Oct 30 19:19:26.484285 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:26.484220    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6lkjf_7563c219-088f-4c66-81f5-86c581f08f34/dns-node-resolver/0.log"
Oct 30 19:19:26.753248 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:26.753160    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-gx2x5_7242bc65-404e-481b-b596-968eb61f0f1c/proxy/0.log"
Oct 30 19:19:26.936515 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:26.936484    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-gx2x5_7242bc65-404e-481b-b596-968eb61f0f1c/proxy/0.log"
Oct 30 19:19:26.963324 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:26.963294    4925 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-47xdf/perf-node-gather-daemonset-zz652"
Oct 30 19:19:27.489425 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:27.489390    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5kvjr_a8ee3e1b-6db3-4036-b26c-d8ad5d4cc8a3/node-ca/0.log"
Oct 30 19:19:28.014839 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:28.014790    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6c45775ff9-c2ghb_c536d86d-2397-47bf-a44f-47b0bf40dd02/manager/0.log"
Oct 30 19:19:28.246978 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:28.246950    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6c45775ff9-c2ghb_c536d86d-2397-47bf-a44f-47b0bf40dd02/manager/0.log"
Oct 30 19:19:29.534809 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:29.534774    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8486858879-ttlr5_e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9/manager/0.log"
Oct 30 19:19:29.805156 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:29.805054    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8486858879-ttlr5_e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9/manager/0.log"
Oct 30 19:19:30.012996 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:30.012954    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-gx2x5_7242bc65-404e-481b-b596-968eb61f0f1c/proxy/0.log"
Oct 30 19:19:30.291455 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:30.291424    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-gx2x5_7242bc65-404e-481b-b596-968eb61f0f1c/proxy/0.log"
Oct 30 19:19:31.593186 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:31.593146    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6c45775ff9-c2ghb_c536d86d-2397-47bf-a44f-47b0bf40dd02/manager/0.log"
Oct 30 19:19:31.790475 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:31.790439    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6c45775ff9-c2ghb_c536d86d-2397-47bf-a44f-47b0bf40dd02/manager/0.log"
Oct 30 19:19:33.662116 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:33.662067    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8486858879-ttlr5_e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9/manager/0.log"
Oct 30 19:19:33.966248 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:33.966172    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8486858879-ttlr5_e1befbbd-48f4-49d2-9ecf-cb6c95d15ab9/manager/0.log"
Oct 30 19:19:34.105269 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:34.105240    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-gx2x5_7242bc65-404e-481b-b596-968eb61f0f1c/proxy/0.log"
Oct 30 19:19:34.410366 ip-10-0-134-58 kubenswrapper[4925]: I1030 19:19:34.410336    4925 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-gx2x5_7242bc65-404e-481b-b596-968eb61f0f1c/proxy/0.log"